Method and apparatus for non-speculative pre-fetch operation in data packet processing

ABSTRACT

A system is provided for enabling a non-speculative pre-fetch operation for processing instructions to be performed in the background ahead of immediate packet processing by a packet processor. The system comprises a packet-management unit for accepting data packets and enqueuing them for processing, a processor unit for processing the data packets, a processor core memory for holding context registers and functional units for processing, a memory for holding a plurality of instruction threads and a software-configurable hardware table for relating queues to pointers to beginnings of instruction threads. The packet-management unit selects an available context in the processor core for processing of a data packet, consults the table, and communicates the pointer to the processor, enabling the processor to perform the non-speculative pre-fetch for instructions.

CROSS-REFERENCE TO RELATED DOCUMENTS

This application is a continuation of U.S. application Ser. No.09/900,393, filed Jul. 5, 2001, now U.S. Pat. No. 7,042,887, which is acontinuation in part (CIP) of U.S. application Ser. No. 09/737,375,filed Dec. 14, 2000, now U.S. Pat. No. 7,058,064, which is includedherein in it's entirety by reference. In addition, application Ser. No.09/737,375 claims priority benefit under 35 U.S.C. 119(e) of ProvisionalPatent Application Ser. No. 60/181,364 filed on Feb. 8, 2000, andincorporates all disclosure of the prior applications by reference.

FIELD OF THE INVENTION

The present invention is in the field of digital processing and pertainsto apparatus and methods for processing packets in routers for packetnetworks, and more particularly to apparatus and methods for enabling anon-speculative pre-fetch operation associated with fetching processinginstruction during packet processing.

BACKGROUND OF THE INVENTION

The well-known Internet network is a notoriously well-knownpublicly-accessible communication network at the time of filing thepresent patent application, and arguably the most robust information andcommunication source ever made available. The Internet is used as aprime example in the present application of a data-packet-network whichwill benefit from the apparatus and methods taught in the present patentapplication, but is just one such network, following a particularstandardized protocol. As is also very well known, the Internet (andrelated networks) are always a work in progress. That is, manyresearchers and developers are competing at all times to provide new andbetter apparatus and methods, including software, for enhancing theoperation of such networks.

In general the most sought-after improvements in data packet networksare those that provide higher speed in routing (more packets per unittime) and better reliability and fidelity in messaging. What isgenerally needed are router apparatus and methods increasing the ratesat which packets may be processed in a router.

As is well-known in the art, packet routers are computerized machineswherein data packets are received at any one or more of typicallymultiple ports, processed in some fashion, and sent out at the same orother ports of the router to continue on to downstream destinations. Asan example of such computerized operations, keeping in mind that theInternet is a vast interconnected network of individual routers,individual routers have to keep track of which external routers to whichthey are connected by communication ports, and of which of alternateroutes through the network are the best routes for incoming packets.Individual routers must also accomplish flow accounting, with a flowgenerally meaning a stream of packets with a common source and enddestination. A general desire is that individual flows follow a commonpath. The skilled artisan will be aware of many such requirements forcomputerized processing.

Typically a router in the Internet network will have one or more CentralProcessing Units (CPUs) as dedicated microprocessors for accomplishingthe many computing tasks required. In the current art at the time of thepresent application, these are single-streaming processors; that is,each processor is capable of processing a single stream of instructions.In some cases developers are applying multiprocessor technology to suchrouting operations. The present inventors have been involved for sometime in development of dynamic multistreaming (DMS) processors, whichprocessors are capable of simultaneously processing multiple instructionstreams. One preferred application for such processors is in theprocessing of packets in packet networks like the Internet.

In a data-packet processor, a configurable queuing system for packetaccounting during processing is known to the inventor. The queuing andaccounting system has a plurality of queues arranged in one or moreclusters, an identification mechanism for creating a packet identifierfor arriving packets, insertion logic for inserting packet identifiersinto queues and for determining into which queue to insert a packetidentifier, and selection logic for selecting packet identifiers fromqueues to initiate processing of identified packets, downloading ofcompleted packets, or for re-queuing of the selected packet identifiers.

One aspect of the above-described queuing system involves selecting andpreloading contexts with packet information for processing and notifyinga processing component of the activation of the context so that theprocessor may fetch an instruction thread or threads to begin andcomplete the processing. Such an operation is typically called aninstruction fetch, or simply a FETCH operation in programming language.

In some prior-art processors, there is a pre-fetch operation knownwherein the processor pre-fetches an instruction thread or threads thatwill “most likely” be required for the processing. Determination forwhich thread or threads to select is speculative in this prior-art case,and in some cases, the selected instruction is not the correctinstruction for the processing of the packet for which it was fetched.The desire to enable such pre-fetch operations stems from an overallgoal of improving the speed of processing for processors in general. If,in the case of a packet processor, which is preferred application forthe present invention, the instructions can be fetched while packetpreparation operations are simultaneously being performed, then thenumber of cycles required to initiate and complete processing of apacket can be reduced. Over multitudes of data packets being processed,this reduction can be significant.

The problem in the prior-art is that the identification and selection ofinstructions during a pre-fetch is speculative, meaning that not enoughinformation is available at the desired point in time where a pre-fetchoperation would be beneficial. Therefore, the pre-fetch operation isspeculative in nature and not reliable in many instances. Logicallythen, the number of cycles required to process a data packet can beincreased over what would normally be the case if a speculativepre-fetch returns incorrect instructions and must then be repeated.

What is clearly needed is a method and apparatus that enables anon-speculative pre-fetch operation wherein correctness of the fetchedinstruction or instructions is assured. Such a system would furtherprovide reduction of cycles required for packet processing and increaseprocessor performance by freeing up other resources for otheroperations.

SUMMARY OF THE INVENTION

In a preferred embodiment of the present invention, in a data-packetprocessor, a system for non-speculative pre-fetching is provided,comprising a processing unit having a first portion for processing thedata packets, using instruction threads specific to packet type, and asecond portion comprising a pool of context registers and functionalunits for processing, a packet-management unit (PMU) for managing datapackets of different types received for processing, including selectingand loading the context registers, a memory storing at least an initialinstruction of instruction threads, and a table equating packet typeswith pointers to memory locations for the at least first instructions ofinstruction threads specific to the packet types. The system ischaracterized in that the PMU selects a context from the pool ofcontexts for processing of a data packet, the table is consulted for thepointer, and the pointer is provided to the processing unit firstportion, enabling the processing unit first portion to prefetch at leastan initial instruction for the packet to be processed at least partiallyin parallel with loading of the context.

In some embodiments the second portion of the processing unit comprisesseparate clusters, each cluster comprising contexts and functionalunits. Also in some embodiments the table is in the PMU. The processormay be a dynamic multi-streaming processor. Also in preferredembodiments the memory holding at least a first instruction of theinstruction threads is an on-chip instruction cache memory, while inothers the memory holding at least a first instruction of theinstruction threads is an off-chip memory.

In some cases data packets to be processed are stored in queuesaccording to instruction threads required to process the packets, andthe queue from which a packet arrives for processing indicates thepacket type.

In another aspect of the invention, in a data-packet processor having afirst portion for processing data packets, using instruction threadsspecific to packet type, and a second portion comprising a pool ofcontext registers and functional units for processing, a method foraccomplishing pre-fetch of at least a first instruction for processingis provided, comprising steps of (a) selecting, by a packet-managementunit (PMU), an available context for loading information for processinga packet ready for processing; (b) consulting a table relating packettype for the packet ready to be processed to a pointer to a memorylocation for at least a first instruction of an instruction thread toprocess the packet; (c) providing the pointer to the first portion; and(d) pre-fetching the at least first instruction of the thread to processthe data packet, at least partially in parallel with loading thecontext.

In some preferred embodiments of the method the second portion of theprocessing unit comprises separate clusters, each cluster comprisingcontexts and functional units. Also in some preferred embodiments thetable is in the PMU. The processor may be a dynamic multi-streamingprocessor. Also in preferred embodiments the memory holding at least afirst instruction of the instruction threads is an on-chip instructioncache memory, while in some other the memory holding at least a firstinstruction of the instruction threads is an off-chip memory. Inpreferred embodiments as well, data packets to be processed are storedin queues according to instruction threads required to process thepackets, and wherein the queue from which a packet arrives forprocessing indicates the packet type.

In embodiments of the invention described in enabling detail below, forthe first time a system and method is provided, useful with dynamicmulti-streaming processors and others, that provides for anon-speculative pre-fetch of instruction threads.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified block diagram showing relationship of functionalareas of a DMS processor in a preferred embodiment of the presentinvention.

FIG. 2 is a block diagram of the DMS processor of FIG. 1 showingadditional detail.

FIG. 3 is a block diagram illustrating uploading of data into the LPM orEPM in an embodiment of the invention.

FIG. 4 a is a diagram illustrating determination and allocation for datauploading in an embodiment of the invention.

FIG. 4 b is a diagram showing the state that needs to be maintained foreach of the four 64 KB blocks.

FIGS. 5 a and 5 b illustrate an example of how atomic pages areallocated in an embodiment of the present invention.

FIGS. 6 a and 6 b illustrate how memory space is efficiently utilized inan embodiment of the invention.

FIG. 7 is a top-level schematic of the blocks of the XCaliber PMU unitinvolved in the downloading of a packet.

FIG. 8 is a diagram illustrating the phenomenon of packet growth andshrink.

FIG. 9 is a block diagram showing high-level communication between theQS and other blocks in the PMU and SPU in an embodiment of the presentinvention.

FIG. 10 is a table illustrating six different modes in an embodiment ofthe invention into which the QS can be configured.

FIG. 11 is a diagram illustrating generic architecture of the QS ofFIGS. 2 and 7 in an embodiment of the present invention.

FIG. 12 is a table indicating coding of the outbound DeviceId field inan embodiment of the invention.

FIG. 13 is a table illustrating priority mapping for RTU transfers in anembodiment of the invention.

FIG. 14 is a table showing allowed combinations of Active, Completed,and Probed bits for a valid packet in an embodiment of the invention.

FIG. 15 is a Pattern Matching Table in an embodiment of the presentinvention.

FIG. 16 illustrates the format of a mask in an embodiment of theinvention.

FIG. 17 shows an example of a pre-load operation using the mask in FIG.16.

FIG. 18 illustrates shows the PMU Configuration Space in an embodimentof the present invention.

FIGS. 19 a, 19 b and 19 c are a table of Configuration register Mapping.

FIG. 20 is an illustration of a PreloadMaskNumber configurationregister.

FIG. 21 illustrates a PatternMatching Table in a preferred embodiment ofthe present invention.

FIG. 22 illustrates a VirtualPageEnable configuration register in anembodiment of the invention.

FIG. 23 illustrates a ContextSpecificPatternMatchingMask configurationregister in an embodiment of the invention.

FIG. 24 illustrates the MaxActivePackets configuration register in anembodiment of the present invention.

FIG. 25 illustrates the TimeCounter configuration register in anembodiment of the present invention.

FIG. 26 illustrates the StatusRegister configuration register in anembodiment of the invention.

FIG. 27 is a schematic of a Command Unit and command queues in anembodiment of the present invention.

FIG. 28 is a table showing the format of command inserted in commandqueues in an embodiment of the present invention.

FIG. 29 is a table showing the format for responses that differentblocks generate back to the CU in an embodiment of the invention.

FIG. 30 shows a performance counter interface between the PMU and theSIU in an embodiment of the invention.

FIG. 31 shows a possible implementation of internal interfaces among thedifferent units in the PMU in an embodiment of the present invention.

FIG. 32 is a diagram of a BypassHooks configuration register in anembodiment of the invention.

FIG. 33 is a diagram of an InternalStateWrite configuration register inan embodiment of the invention.

FIGS. 34-39 comprise a table listing events related to performancecounters in an embodiment of the invention.

FIG. 40 is a table illustrating the different bypass hooks implementedin the PMU in an embodiment of the invention.

FIG. 41 is a table relating architecture and hardware blocks in anembodiment of the present invention.

FIGS. 42-45 comprise a table showing SPU-PMU Interface in an embodimentof the invention.

FIGS. 46-49 comprise a table showing SIU-PMU Interface in an embodimentof the invention.

FIG. 50 is a block-diagram logically illustrating components andinteraction during a pre-fetch operation according to an embodiment ofthe present invention.

FIG. 51 is a flow chart illustrating general steps for initiating andcompleting a non-speculative pre-fetch operation according to anembodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the provisional patent application Ser. No. 60/181,364 referencedabove there is disclosure as to the architecture of a DMS processor,termed by the inventors the XCaliber processor, which is dedicated topacket processing in packet networks. Two extensive diagrams areprovided in the referenced disclosure, one, labeled NIO Block Diagram,shows the overall architecture of the XCaliber processor, with input andoutput ports to and from a packet-handling ASIC, and the otherillustrates numerous aspects of the Generic Queue shown in the NIOdiagram. The NIO system in the priority document equates to the PacketManagement Unit (PMU) in the present specification. It is to the severalaspects of the generic queue that the present application is directed.

FIG. 1 is a simplified block diagram of an XCaliber DMS processor 101with a higher-level subdivision of functional units than that shown inthe NIO diagram of the priority document. In FIG. 1 XCaliber DMSprocessor 101 is shown as organized into three functional areas. Anoutside System Interface Unit (SIU) area 107 provides communication withoutside devices, that is, external to the XCaliber processor, typicallyfor receiving and sending packets. Inside, processor 101 is divided intotwo broad functional units, a Packet Management Unit (PMU) 103, equatingto the NIO system in the priority document mentioned above, and a StreamProcessor Unit (SPU) 107. The functions of the PMU include accountingfor and managing all packets received and processed. The SPU isresponsible for all computational tasks.

The PMU is a part of the XCaliber processor that offloads the SPU fromperforming costly packet header accesses and packet sorting andmanagement tasks, which would otherwise seriously degrade performance ofthe overall processor.

Packet management is achieved by (a) Managing on-chip memory allocatedfor packet storage, (b) Uploading, in the background, packet headerinformation from incoming packets into different contexts (contextregisters, described further below) of the XCaliber processor, (c)Maintaining, in a flexible queuing system, packet identifiers of thepackets currently in process in the XCaliber.

The described packet management and accounting tasks performed by thePMU are performed in parallel with processing of packets by the SPUcore. To implement this functionality, the PMU has a set of hardwarestructures to buffer packets incoming from the network, provide them tothe SPU core and, if needed, send them out to the network when theprocessing is completed. The PMU features a high degree ofprogrammability of several of its functions, such as configuration ofits internal packet memory storage and a queuing system, which is afocus of the present patent application.

FIG. 2 is a block diagram of the XCaliber processor of FIG. 1 showingadditional detail. SIU 107 and SPU 105 are shown in FIG. 2 as singleblocks with the same element numbers used in FIG. 1. The PMU is shown inconsiderably expanded detail, however, with communication lines shownbetween elements.

In FIG. 2 there is shown a Network/Switching Fabric Interface 203 whichis in some cases an Application Specific Integrated Circuit (ASIC)dedicated for interfacing directly to a network, such as the Internetfor example, or to switching fabric in a packet router, for example,receiving and transmitting packets, and transacting the packets with theXCaliber processor. In this particular instance there are two in portsand two out ports communicating with processor 201. Network in and outinterface circuitry 205 and 215 handle packet traffic onto and off theprocessor, and these two interfaces are properly a part of SIU 107,although they are shown separately in FIG. 2 for convenience.

Also at the network interface within the PMU there are, in processor201, input and output buffers 207 and 217 which serve to buffer the flowof packets into and out of processor 201.

Referring again to FIG. 1, there is shown a Packet Management Unit (PMU)103, which has been described as a unit that offloads the requirementfor packet management and accounting from the Stream Processing Unit.This is in particular the unit that has been expanded in FIG. 2, andconsists substantially of Input Buffer (IB) 207, Output Buffer (OB) 217,Paging Memory Management Unit (PMMU) 209, Local Packet Memory (LPM) 219,Command Unit (CU) 213, Queueing System (QS) 211, Configuration Registers221, and Register Transfer Unit (RTU) 227. The communication pathsbetween elements of the PMU are indicated by arrows in FIG. 2, andfurther description of the elements of the PMU is provided below,including especially QS 211, which is a particular focus of the presentpatent application.

Overview of PMU

Again, FIG. 2 shows the elements of the PMU, which are identifiedbriefly above. Packets arrive to the PMU in the present example througha 16-byte network input interface. In this embodiment packet dataarrives to the PMU at a rate of 20 Gbps (max). At an operating speed of300 MHz XCaliber core frequency, an average of 8 bytes of packet dataare received every XCaliber core cycle. The incoming data from thenetwork input interface is buffered in InBuffer (IB) block 207. Networkinterface 205 within XCaliber has the capability of appending to thepacket itself the size of the packet being sent, in the event that theexternal device has not been able to append the size to the packetbefore sending the packet. Up to 2 devices can send packet data toXCaliber at (10 Gbps per device), and two in ports are shown from anattached ASIC. It is to be understood that the existence and use of theparticular ASIC is exemplary, and packets could be received from otherdevices. Further, there may be in some embodiments more or fewer thanthe two in ports indicated.

Packet Memory Manager Unit (PMMU) 209 decides whether each incomingpacket has to be stored into on-chip Local Packet Memory (LPM) 219, or,in the case that, for example, no space exists in the LPM to store it,may decide to either send the packet out to an External Packet Memory(EPM) not shown through the SIU block, or may decide to drop the packet.In case the packet is to be stored in the LPM, the PMMU decides where tostore the packet and generates all the addresses needed to do so. Theaddresses generated correspond in a preferred embodiment to 16-bytelines in the LPM, and the packet is consecutively stored in this memory.

In the (most likely) case that the PMMU does not drop the incomingpacket, a packet identifier is created, which includes a pointer (namedpacketPage) to a fixed-size page in packet memory where the packet hasstarted to be stored. The identifier is created and enqueued intoQueuing System (QS) block 211. The QS assigns a number from 0 to 255(named packetNumber) to each new packet. The QS sorts the identifiers ofthe packets alive in XCaliber based on the priority of the packets, andit updates the sorting when the SPU core notifies any change on thestatus of a packet. The QS selects which packet identifiers will beprovided next to the SPU. Again, the QS is a particular focus of thepresent application.

Register Transfer Unit (RTU) block 227, upon receiving a packetidentifier (packetPage and packetNumber) from the QS, searches for anavailable context (229, FIG. 2) out of 8 contexts that XCaliber featuresin a preferred embodiment. For architectural and description purposesthe contexts are considered a part of a broader Stream Processing Unit,although the contexts are shown in FIG. 2 as a separate unit 229.

In the case that no context is available, the RTU has the ability tonotify the SPU about this event through a set of interrupts. In the casethat a context is available, the RTU loads the packet identifierinformation and some selected fields of the header of the packet intothe context, and afterwards it releases the context (which will at thattime come under control of the SPU. The RTU accesses the headerinformation of the packet through the SIU, since the packet could havebeen stored in the off-chip EPM.

Eventually a stream in the SPU core processes the context and notifiesthe QS of this fact. There are, in a preferred embodiment, eight streamsin the DMS core. The QS then updates the status of the packet (tocompleted), and eventually this packet is selected for downloading (i.e.the packet data of the corresponding packet is sent out of the XCaliberprocessor to one of the two external devices).

When a packet is selected for downloading, the QS sends the packetPage(among other information) to the PMMU block, which generates thecorresponding line addresses to read the packet data from the LPM (incase the packet was stored in the on-chip local memory) or it willinstruct the SIU to bring the packet from the external packet memory tothe PMU. In any case, the lines of packet data read are buffered intothe OutBuffer (OB) block, and from there sent out to the device throughthe 16-byte network output interface. This interface is independent ofits input counterpart. The maximum aggregated bandwidth of thisinterface in a preferred embodiment is also 20 Gbps, 10 Gbps per outputdevice.

CommandUnit (CU) 213 receives commands sent by SPU 105. A commandcorresponds to a packet instruction, which are in many cases newlydefined instructions, dispatched by the SPU core. These commands aredivided into three independent types, and the PMU can execute onecommand per type per cycle (for a total of up to 3 commands per cycle).Commands can be load-like or store-like (depending on whether the PMUprovides a response back to the SPU or not, respectively).

A large number of features of the PMU are configured by the SPU throughmemory-mapped configuration registers 221. Some such features have to beprogrammed at boot time, and the rest can be dynamically changed. Forsome of the latter, the SPU has to be running in a single-thread mode toproperly program the functionality of the feature. The CU block managesthe update of these configuration registers.

The PMU provides a mechanism to aid in flow control between ASIC 203 andXCaliber DMS processor 201. Two different interrupts are generated bythe PMU to SPU 105 when LPM 219 or QS 211 are becoming full. Softwarecontrols how much in advance the interrupt is generated before thecorresponding structure becomes completely full. Software can alsodisable the generation of these interrupts.

LPM 219 is also memory mapped, and SPU 105 can access it through theconventional load/store mechanism. Both configuration registers 221 andLPM 219 have a starting address (base address) kept by SIU 107. Requestsfrom SPU 105 to LPM 219 and the configuration space arrive to the PMUthrough SIU block 107. The SIU is also aware of the base address of theexternal packet memory.

In Buffer (IB)

Packet data sent by an external device arrives to the PMU through thenetwork input interface 205 at an average rate of 8 bytes every XCalibercore cycle in a preferred embodiment. IB block 207 of the PMU receivesthis data, buffers it, and provides it, in a FIFO-like fashion, to LPM219 and in some cases also to the SIU (in case of a packet overflow, asexplained elsewhere in this specification.

XCaliber DMS processor 201 can potentially send/receive packet datato/from up to 2 independent devices. Each device is tagged in SIU 107with a device identifier, which is provided along with the packet data.When one device starts sending data from a packet, it will continue tosend data from that very same packet until the end of the packet isreached or a bus error is detected by the SIU.

In a preferred embodiment the first byte of a packet always starts atbyte 0 of the first 16 bytes sent of that packet. The first two bytes ofthe packet specify the size in bytes of the packet (including thesefirst two bytes). These two bytes are always appended by the SIU if theexternal device has not appended them. If byte k in the 16-byte chunk isa valid byte, bytes 0 . . . k-1 are also valid bytes. This can beguaranteed since the first byte of a packet always starts at byte 0.Note that no valid bits are needed to validate each byte since a packetalways starts at byte 0 of the 16-byte chunk, and the size of the packetis known up front (in the first two bytes). The network interfaceprovides, at every core clock, a control bit specifying whether the16-byte chunk contains, at least, one valid byte.

The valid data received from the network input interface is organized inbuffer 207. This is an 8-entry buffer, each entry holding the 16-bytesof data plus the control bits associated to each chunk. PMMU 209 looksat the control bits in each entry and determines whether a new packetstarts or to which of the (up to) two active packets the data belongsto, and it acts accordingly.

The 16-byte chunks in each of the entries in IB 207 are stored in LPM219 or in the EPM (not shown). It is guaranteed by either the LPMcontroller or the SIU that the bandwidth to write into the packet memorywill at least match the bandwidth of the incoming packet data, and thatthe writing of the incoming packet data into the packet memory will havehigher priority over other accesses to the packet memory.

In some cases IB 207 may get full because PMMU 209 may be stalled, andtherefore the LPM will not consume any more data of the IB until thestall is resolved. Whenever the IB gets full, a signal is sent tonetwork input interface 205, which will retransmit the next 16-bytechunk as many times as needed until the IB accepts it. Thus, no packetdata is lost due to the IB getting full.

Out Buffer (OB)

Network output interface 215 also supports a total aggregated bandwithof 20 Gbps (10 Gbps per output device), as does the Input Interface. At300 MHz XCaliber clock frequency, the network output interface acceptsin average 8 bytes of data every XCaliber cycle from the OB block, andsends it to one of the two output devices. The network input and outputinterfaces are completely independent of each other.

Up to 2 packets (one per output device) can be simultaneously sent. Thedevice to which the packet is sent does not need to correspond to thedevice that sent the packet in. The packet data to be sent out will comefrom either LPM 219 or the EPM (not shown).

For each of the two output devices connected at Network Out interface215, PMMU 209 can have a packet ready to start being downloaded, apacket being downloaded, or no packet to download. Every cycle PMMU 209selects the highest packet across both output devices and initiates thedownload of 16 bytes of data for that packet. Whenever the PMMU isdownloading packet data from a packet to an output device, no data froma different packet will be downloaded to the same device until thecurrent packet is completely downloaded.

The 16-byte chunks of packet data read from LPM 219 (along with someassociated control information) are fed into one of the two 8-entrybuffers (one per device identifier). The contents of the head of one ofthese buffers is provided to the network output interface whenever thisinterface requests it. When the head of both buffers is valid, the OBprovides the data in a round robin fashion.

Differently than the network input interface, in the 16-byte chunk sentto the network output interface it can not be guaranteed that if a bytek is valid, then bytes 0 . . . k-1 are valid as well. The reason forthis is that when the packet is being sent out, it does not need tostart at byte 0 of the 16-byte chunk in memory. Thus, for each 16-bytechunk of data that contains the start of the packet to be sent out, OB217 needs to notify the network interface where the first valid byte ofthe chunk resides. Moreover, since the first two bytes of the packetcontain the size of the packet in bytes, the network output interfacehas the information to figure out where the last valid byte of thepacket resides within the last 16-byte chunk of data for that packet.Moreover, OB 217 also provides a control bit that informs SIU 107whether it needs to compute CRC for the packet, and if so, which type ofCRC. This control bit is provided by PMMU 209 to OB 217.

Paging Memory Management Unit (PMMU)

The packet memory address space is 16 MB. Out of the 16 MB, the XCaliberprocessor features 256 KB on-chip. The rest (or a fraction) isimplemented using external storage.

The packet memory address space can be mapped in the TLB of SPU 105 asuser or kernel space, and as cachable or uncachable. In case it ismapped cachable, the packet memory space is cached (write-through) intoan L1 data cache of SPU 105, but not into an L2 cache.

A goal of PMMU 209 is to store incoming packets (and SPU-generatedpackets as well) into the packet memory. In case a packet from thenetwork input interface fits into LPM 219, PMMU 209 decides where tostore it and generates the necessary write accesses to LPM 219; in casethe packet from the network input interface is going to be stored in theEPM, SPU 105 decides where in the EPM the packet needs to be stored andSIU 107 is in charge of storing the packet. In either case, the packetis consecutively stored and a packet identifier is created by PMMU 209and sent to QS 211.

SPU 105 can configure LPM 219 so packets larger than a given size willnever be stored in the LPM. Such packets, as well as packets that do notfit into the LPM because lack of space, are sent by PMMU 209 to the EPMthrough SIU 107. This is a mechanism called overflow and is configuredby the SPU for the PMU to do so. If no overflow of packets is allowed,then the packet is dropped. In this case, PMMU 209 interrupts the SPU(again, if configured to do so).

Uploading a Packet into Packet Memory

Whenever there is valid data at the head of IB 205, the correspondingdevice identifier bit is used to determine to which packet (out of thetwo possible packets being received) the data belongs. When the networkinput interface starts sending data of a new packet with deviceidentifier d, all the rest of the data will eventually arrive with thatsame device identifier d unless an error is notified by the networkinterface block. The network input interface can interleave data fromtwo different device identifiers, but in a given cycle only data fromone device is received by IB 207.

When a packet needs to be stored into LPM 219, PMMU block 209 generatesall the write addresses and write strobes to LPM 219. If the packetneeds to be stored into the EPM, SIU 107 generates them.

FIG. 3 is a diagram illustrating uploading of data into either LPM 219or the EPM, which is shown in FIG. 3 as element 305, but not shown inFIG. 2. The write strobe to the LPM or EPM will not be generated unlessthe header of the IB has valid data. Whenever the write strobe isgenerated, the 16-byte chunk of data at the head of the IB (whichcorresponds to a LPM line) is deleted from the IB and stored in the LPMor EPM. The device identifier bit of the head of the IB is used toselect the correct write address out of the 2 address generators (oneper input device).

In the current embodiment only one incoming packet can be simultaneouslystored in the EPM by the SIU (i.e. only one overflow packet can behandled by the SIU at a time). Therefore, if a second packet that needsto be overflowed is sent by the network input interface, the data ofthis packet will be thrown away (i.e. the packet will be dropped).

A Two Byte Packet-Size Header

The network input interface always appends two bytes to a packetreceived from the external device (unless this external device alreadydoes so, in which case the SIU will be programmed not to append them).This appended data indicates the size in bytes of the total packet,including the two appended bytes. Thus, the maximum size of a packetthat is processed by the XCaliber DMS processor is 65535 bytes includingthe first two bytes.

The network output interface expects that, when the packet is returnedby the PMU (if not dropped during its processing), the first two bytesalso indicate the size of the processed packet. The size of the originalpacket can change (the packet can increase or shrink) as a result ofprocessing performed by the XCaliber processor. Thus, if the processingresults in increasing the size beyond 64K-1 bytes, it is theresponsibility of software to chop the packet into two different smallerpackets.

The PMU is more efficient when the priority of the packet being receivedis known up front. The third byte of the packet will be used forpriority purpose if the external device is capable of providing thisinformation to the PMU. The software programs the PMU to either use theinformation in this byte or not, which is does through a boot-timeconfiguration register named Log2InQueues.

Dropping a Packet

A packet completely stored in either LPM 219 or EPM 305 will be droppedonly if SPU 105 sends an explicit command to the PMU to do so. Noautomatic dropping of packets already stored in the packet memory canoccur. In other words, any dropping algorithm of packets received by theXCaliber DMS processor is implemented in software.

There are, however, several situations wherein the PMU may drop anincoming packet. These are (a) The packet does not fit in the LPM andthe overflow of packets is disabled, (b) The total amount of bytesreceived for the packet is not the same as the number of bytes specifiedby the ASIC in the first two bytes of the ASIC-specific header, or (c) Atransmission error has occurred between the external device and thenetwork input interface block of the SIU. The PMMU block is notifiedabout such an error.

For each of the cases (a), (b) and (c) above, an interrupt is generatedto the SPU. The software can disable the generation of these interruptsusing AutomaticPacketDropIntEnable, PacketErrorIntEnable on-the-flyconfiguration flags.

Virtual Pages

An important process of PMMU 209 is to provide an efficient way toconsecutively store packets into LPM 219 with as little memoryfragmentation as possible. The architecture in the preferred embodimentprovides SPU 105 with a capability of grouping, as much as possible,packets of similar size in the same region of LPM 219. This reducesoverall memory fragmentation.

To implement the low-fragmentation feature, LPM 219 is logically dividedinto 4 blocks of 64 KB bytes each. Each block is divided into fixedatomic pages of 256 bytes. However, every block has virtual pages thatrange from 256 bytes up to 64 KB, in power-of-2 increments. Software canenable/disable the different sizes of the virtual pages for each of the4 blocks using an on-the-fly configuration register namedVirtualPageEnable. This allows configuring some blocks to store packetsof up to a certain size.

The organization and features of the PMU assure that a packet of size swill never be stored in a block with a maximum virtual page size lessthan s. However, a block with a minimum virtual page size of r willaccept packets of size smaller than r. This will usually be the case,for example, in which another block or blocks are configured to storethese smaller packets, but is full.

Software can get ownership of any of the four blocks of the LPM, whichimplies that the corresponding 64 KB of memory will become softwaremanaged. A configuration flag exists per block (SoftwareOwned) for thispurpose. The PMMU block will not store any incoming packet from thenetwork input interface into a block in the LPM with the associatedSoftwareOwned flag asserted. Similarly, the PMMU will not satisfy aGetSpace operation (described elsewhere) with memory of a block with itsSoftwareOwned flag asserted. The PMMU, however, is able to download anypacket stored by software in a software-owned block.

The PMMU logic determines whether an incoming packet fits in any of theblocks of the LPM. If a packet fits, the PMMU decides in which of thefour blocks (since the packet may fit in more than one block), and thefirst and last atomic page that the packet will use in the selectedblock. The atomic pages are allocated for the incoming packet. Whenpacket data stored in an atomic page has been safely sent out of theXCaliber processor through the network output interface, thecorresponding space in the LPM can be de-allocated (i.e. made availablefor other incoming packets).

The EPM, like the LPM is also logically divided into atomic pages of 256bytes. However, the PMMU does not maintain the allocation status ofthese pages. The allocation status of these pages is managed bysoftware. Regardless of where the packet is stored, the PMMU generatesan offset (in atomic pages) within the packet memory to where the firstdata of the packet is stored. This offset is named henceforthpacketPage. Since the maximum size of the packet memory is 16 MB, thepacketPage is a 16-bit value.

As soon as the PMMU safely stores the packet in the LPM, or receivesacknowledgement from SIU 107 that the last byte of the packet has beensafely stored in the EPM, the packetPage created for that packet is sentto the QS. Operations of the QS are described in enabling detail below.

Generating the packetPage Offset

The PMMU always monitors the device identifier (deviceId) associated tothe packet data at the head of the IB. If the deviceId is not currentlyactive (i.e. the previous packet sent by that deviceId has beencompletely received), that indicates that the head of the IB containsthe first data of a new packet. In this case, the first two bytes (byte0and byte1 in the 16-byte chunk) specify the size of the packet in bytes.With the information of the size of the new incoming packet, the PMMUdetermines whether the packet fits into LPM 219 and, if it does, inwhich of the four blocks it will be stored, plus the starting and endingatomic pages within that block.

The required throughput in the current embodiment of the PMMU todetermine whether a packet fits in LPM 219 and, if so, which atomicpages are needed, is one packet every two cycles. One possible two-cycleimplementation is as follows: (a) The determination happens in onecycle, and only one determination happens at a time (b) In the cyclefollowing the determination, the atomic pages needed to store the packetare allocated and the new state (allocated/de-allocated) of the virtualpages are computed. In this cycle, no determination is allowed.

FIG. 4 a is a diagram illustrating determination and allocation inparallel for local packet memory. The determination logic is performedin parallel for all of the four 64 KB blocks as shown.

FIG. 4 b shows the state that needs to be maintained for each of thefour 64 KB blocks. This state, named AllocationMatrix, is recomputedevery time one or more atomic pages are allocated or de-allocated, andit is an input for the determination logic. The FitsVector andIndexVector contain information computed from the AllocationMatrix.

AllocationMatrix[VPSize][VPIndex] indicates whether virtual page numberVPIndex of size VPSize in bytes is already allocated or not.FitsVector[VPSize] indicates whether the block has at least onenon-allocated virtual page of size VPSize. If FitsVector[VPSize] isasserted, IndexVector[VPSize] vector contains the index of anon-allocated virtual page of size VPSize.

The SPU programs which virtual page sizes are enabled for each of theblocks. The EnableVector[VPSize] contains this information. Thisconfiguration is performed using the VirtualPageEnable on-the-flyconfiguration register. Note that the AllocationMatrix[][],FitsVector[], IndexVector[] and EnableVector[] are don't cares if thecorresponding SoftwareOwned flag is asserted.

In this example the algorithm for the determination logic (for a packetof size s bytes) is as follows:

-   1) Fits logic: check, for each of the blocks, whether the packet    fits in or not. If it fits, remember the virtual page size and the    number of the first virtual page of that size.

For All Block j Do (can be done in parallel): Fits[j] = (s <= VPSize)AND FitsVector[VPSize] AND Not SoftwareOwned where VPSize is thesmallest possible page size. If (Fits[j]) VPIndex[j] =IndexVector[VPSize] MinVPS[j] = VPSize Else MinVPS[j] = <Infinity>

-   2) Block selection: the blocks with the smallest virtual page    (enabled or not) that is able to fit the packet in are candidates.    The block with the smallest enabled virtual page is selected.

If Fits[j] = FALSE for all j Then <Packet does not fit in LPM>packetPage = OverflowAddress >> 8 Else C = set of blocks with smallestMinVPS AND Fits[MinVPS] B = block# in C with the smallest enabledvirtual page (if more than one exists, pick the smallest block number)If one or more blocks in C have virtual pages enabled Then Index =VPIndex[B] VPSize = MinVPS[B] NumAPs = ceil(S/256) packetPage =(B*64KB + Index*VPSize)>> Else <Packet does not fit in LPM> packetPage =OverflowAddress >> 8

If the packet fits in the LPM, the packetPage created is then the atomicpage number within the LPM (there are up to 1K different atomic pages inthe LPM) into which the first data of the packet is stored. If thepacket does not fit, then the packetPage is the contents of theconfiguration register OverflowAddress right-shifted 8 bits. The packetoverflow mechanism is described elsewhere in this specification, with asubheader “Packet overflow”.

In the cycle following the determination of where the packet will bestored, the new values of the AllocationMatrix, FitsVector andIndexVector must be recomputed for the selected block. IfFitsVector[VPSize] is asserted, then IndexVector[VPSize] is the index ofthe largest non-allocated virtual page possible for the correspondingvirtual page size. If FitsVector[VPSize] is de-asserted, thenIndexVector[VPSize] is undefined.

The number of atomic pages needed to store the packet is calculated(NumAPs) and the corresponding atomic pages are allocated. Theallocation of the atomic pages for the selected block (B) is done asfollows:

-   1. The allocation status of the atomic pages in    AllocationMatrix[APsize][j . . . k], j being the first atomic page    and k the last one (k−j+1=NumAPs), are set to allocated.-   2. The allocation status of the virtual pages in    AllocationMatrix[r][s] are updated following the mesh structure in    FIG. 4 b. (a 2^(k+1)-byte virtual page will be allocated if any of    the two 2^(k)-byte virtual pages that it is composed of is    allocated).

When the packetPage has been generated, it is sent to the QS forenqueueing. If the QS is full (very rare), it will not be able to acceptthe packetPage being provided by the PMMU. In this case, the PMMU willnot be able to generate a new packetPage for the next new packet. Thisputs pressure on the IB, which might get full if the QS remains full forseveral cycles.

The PMMU block also sends the queue number into which the QS has tostore the packetPage. How the PMMU generates this queue number isdescribed below in sections specifically allocated to the QS.

Page Allocation Example

FIGS. 5 a and 5 b illustrate an example of how atomic pages areallocated. For simplicity, the example assumes 2 blocks (0 and 1) of 2KB each, with an Atomic page size of 256 bytes, and both blocks havetheir SoftwareOwned flag de-asserted. Single and double cross-hatchedareas represent allocated virtual pages (single cross-hatched pagescorrespond to the pages being allocated in the current cycle). Theexample shows how the pages get allocated for a sequence of packet sizesof 256, 512, 1K and 512 bytes. Note that, after this sequence, a 2K-bytepacket, for example, will not fit in the example LPM.

Whenever the FitsVector[VPSize] is asserted, the IndexVector[VPSize]contains the largest non-allocated virtual page index for virtual pagesize VPSize. The reason for choosing the largest index is that thememory space is better utilized. This is shown in FIGS. 6 a and 6 b,where two 256-byte packets are stored in a block. In scenario A, the256-byte virtual page is randomly chosen, whereas in scenario B, thelargest index is always chosen. As can be seen, the block in scenario Aonly allows two 512-byte virtual pages, whereas the block in scenario Ballows three. Both, however, allow the same number of 256-byte packetssince this is the smallest allocation unit. Note that the same effect isobtained by choosing the smallest virtual page index number all thetime.

Packet Overflow

The only two reasons why a packet cannot be stored in the LPM are (a)that the size of the packet is larger than the maximum virtual pageenabled across all 4 blocks; or (b) that the size of the packet issmaller than or equal to the maximum virtual page enabled but no spacecould be found in the LPM.

When a packet does not fit into the LPM, the PMMU will overflow thepacket through the SIU into the EPM. To do so, the PMMU provides theinitial address to the SIU (16-byte offset within the packet memory) towhere the packet will be stored. This 20-bit address is obtained asfollows: (a) The 16 MSB bits correspond to the 16 MSB bits of theOverflowAddress configuration register (i.e. the atomic page numberwithin the packet memory). (b) The 4 LSB bits correspond to theHeaderGrowthOffset configuration register. The packetPage value (whichwill be sent to the QS) for this overflowed packet is then the 16 MSBbits of the OverflowAddress configuration register.

If the on-the-fly configuration flag OverflowEnable is asserted, thePMMU will generate an OverflowStartedInt interrupt. When theOverflowStartedInt interrupt is generated, the size in bytes of thepacket to overflow is written by the PMMU into the SPU-read-onlyconfiguration register SizeOfOverflowedPacket. At this point, the PMMUsets an internal lock flag that will prevent a new packet fromoverflowing. This lock flag is reset when the software writes into theon-the-fly configuration register OverflowAddress. If a packet needs tobe overflowed but the lock flag is set, the packet will be dropped.

With this mechanism, it is guaranteed that only one interrupt will begenerated and serviced per packet that is overflowed. This also createsa platform for software to decide where the starting address into whichthe next packet that will be overflowed will be stored is visible to theinterrupt service routine through the SizeOfOverflowedPacket register.In other words, software manages the EPM.

If software writes the OverflowAddress multiple times in between twoOverflowStartedInt interrupts, the results are undefined. Moreover, ifsoftware sets the 16 MSB bits of OverflowAddress to 0 . . . 1023,results are also undefined since the first 1K atomic pages in the packetmemory correspond to the LPM.

Downloading a Packet from Packet Memory

Eventually the SPU will complete the processing of a packet and willinform the QS of the fact. At this point the packet may be downloadedfrom memory, either LPM or EPM, and sent, via the OB to one of theconnected devices. FIG. 7 is a top-level schematic of the blocks of theXCaliber DMS processor involved in the downloading of a packet, and theelements in FIG. 7 are numbered the same as in FIG. 2. The downloadingprocess may be followed in FIG. 7 with the aid of the followingdescriptions.

When QS 211 is informed that processing of a packet is complete, the QSmarks this packet as completed and, a few cycles later (depending on thepriority of the packet), the QS provides to PMMU 209 (as long as thePMMU has requested it) the following information regarding the packet:

(a) the packetPage

(b) the priority (cluster number from which it was extracted)

(c) the tail growth/shrink information (described later in spec)

(d) the outbound device identifier bit

(e) the CRC type field (described later in spec)

(f) the KeepSpace bit

The device identifier sent to PMMU block 209 is a 1-bit value thatspecifies the external device to which the packet will be sent. Thisoutbound device identifier is provided by software to QS 211 as a 2-bitvalue.

If the packet was stored in LPM 219, PMMU 209 generates all of the(16-byte line) read addresses and read strobes to LPM 219. The readstrobes are generated as soon as the read address is computed and thereis enough space in OB 217 to buffer the line read from LPM 219. Bufferdin the OB is associated to device identifier d. This buffer may becomefull for either two reasons: (a) The external device d temporarily doesnot accept data from XCaliber; or (b) The rate of reading data from theOB is lower than the rate of writing data into it.

As soon as the packet data within an atomic page has all been downloadedand sent to the OB, that atomic page can be de-allocated. Thede-allocation of one or more atomic pages follows the same procedure asdescribed above. However, no de-allocation of atomic pages occurs if theLPM bit is de-asserted. The KeepSpace bit is a don't care if the packetresides in EPM 701.

If the packet was stored in EPM 701, PMMU 209 provides to SIU 107 theaddress within the EPM where the first byte of the packet resides. TheSIU performs the downloading of the packet from the EPM. The SIU alsomonitors the buffer space in the corresponding buffer in OB 217 todetermine whether it has space to write the 16-byte chunk read from EPM701. When the packet is fully downloaded, the SIU informs the PMMU ofthe fact so that the PMMU can download the next packet with the samedevice identifier.

When two packets (one per device) are being simultaneously sent, datafrom the packet with highest priority is read out of the memory first.This preemption can happen at a 16-byte boundary or when the packetfinishes its transmission. If both packets have the same priority(provided by the QS), a round-robin method is used to select the packetfrom which data will be downloaded next. This selection logic also takesinto account how full the two buffers in the OB are. If buffer d isfull, for example, no packet with a device identifier d will be selectedin the PMMU for downloading the next 16-byte chunk of data.

When a packet starts to be downloaded from the packet memory (local orexternal), the PMMU knows where the first valid byte of the packetresides. However, the packet's size is not known until the first line(or the first two lines in some cases) of packet data is read from thepacket memory, since the size of the packet resides in the first twobytes of the packet data. Therefore, the processing of downloading apacket first generates the necessary line addresses to determine thesize of the packet, and then, if needed, generates the rest of theaccesses.

This logic takes into account that the first two bytes that specify thesize of the packet can reside in any position in the 16-byte line ofdata. A particular case is when the first two bytes span two consecutivelines (which will occur when the first byte is the 16th byte of a line,and second byte is the 1^(st) byte of next line.

As soon as the PMMU finishes downloading a packet (all the data of thatpacket has been read from packet memory and sent to OB), the PMMUnotifies the QS of this event. The QS then invalidates the correspondingpacket from its queuing system.

When a packet starts to be downloaded, it cannot be preempted, i.e. thepacket will finish its transmission. Other packets that become ready tobe downloaded with the same outbound device identifier while theprevious packet is being transmitted cannot be transmitted until theprevious packet is fully transmitted.

Packet Growth/Shrink

As a result of processing a packet, the size of a network packet cangrow, shrink or remain the same size. If the size varies, the SPU has towrite the new size of the packet in the same first two bytes of thepacket. The phenomenon of packet growth and shrink is illustrated inFIG. 8.

Both the header and the tail of the packet can grow or shrink. When apacket grows, the added data can overwrite the data of another packetthat may have been stored right above the packet experiencing headergrowth, or that was stored right below in the case of tail growth. Toavoid this problem the PMU can be configured so that an empty space isallocated at the front and at the end of every packet when it is storedin the packet memory. These empty spaces are specified withHeaderGrowthOffset and TailGrowthOffset boot-time configurationregisters, respectively, and their granularity is 16 bytes. The maximumHeaderGrowthOffset is 240 bytes (15 16-byte chunks), and the maximumTailGrowthOffset is 1008 bytes (63 16-byte chunks). The minimum in bothcases is 0 bytes. Note that these growth offsets apply to all incomingpackets, that is, there is no mechanism to apply different growthoffsets to different packets.

When the PMMU searches for space in the LPM, it will look for contiguousspace of Size(packet)+((HeaderGrowthOffset+TailGrowthOffset)<<4). Thus,the first byte of the packet (first byte of the ASIC-specific header)will really start at offset ((packetPage<<8)+(HeaderGrowthOffset<<4))within the packet memory.

The software knows what the default offsets are, and, therefore, knowshow much the packet can safely grow at both the head and the tail. Incase the packet needs to grow more than the maximum offsets, thesoftware has to explicitly move the packet to a new location in thepacket memory. The steps to do this are as follows:

-   1) The software requests the PMU for a chunk of contiguous space of    the new size. The PMU will return a new packetPage that identifies    (points to) this new space.-   2) The software writes the data into the new memory space.-   3) The software renames the old packetPage with the new packetPage.-   4) The software requests the PMU to de-allocate the space associated    to the old packetPage.

In the case of header growth or shrinkage, the packet data will nolonger start at ((packetPage<<8)+(HeaderGrowthOffset<<4)). The newstarting location is provided to the PMU with a special instructionexecuted by the SPU when the processing of the packet is completed. Thisinformation is provided to the PMMU by the QS block.

Time Stamp

The QS block of the PMU (described in detail in a following section)guarantees the order of the incoming packets by keeping the packetPageidentifiers of the packets in process in the XCaliber processor inFIFO-like queues. However, software may break this ordering byexplicitly extracting identifiers from the QS, and inserting them at thetail of any of the queues.

To help software in guaranteeing the relative order of packets, the PMUcan be configured to time stamp every packet that arrives to the PMMUblock using an on-the-fly configuration flag TimeStampEnabled. The timestamp is an 8-byte value, obtained from a 64-bit counter that isincremented every core clock cycle.

When the time stamp feature is on, the PMMU appends the 8-byte timestamp value in front of each packet, and the time stamp is stripped offwhen the packet is sent to the network output interface. The time stampvalue always occupies the 8 MSB bytes of the (k-1)th 16-byte chunk ofthe packet memory, where k is the 16-byte line offset where the data ofthe packet starts (k>0). In the case that HeaderGrowthOffset is 0, thetime stamp value will not be appended, even if TimeStampEnabled isasserted.

The full 64-bit time counter value is provided to software through aread-only configuration register (TimeCounter).

Software Operations on the PMMU

Software has access to the PMMU to request or free a chunk of contiguousspace. In particular, there are two operations that software can performon the PMMU. Firstly the software, through an operation GetSpace(size),may try to find a contiguous space in the LPM for size bytes. The PMUreplies with the atomic page number where the contiguous space that hasbeen found starts (i.e. the packetPage), and a success bit. If the PMUwas able to find space, the success bit is set to ‘1’, otherwise it isset to ‘0’. GetSpace will not be satisfied with memory of a block thathas its SoftwareOwned configuration bit asserted. Thus, softwareexplicitly manages the memory space of software-owned LPM blocks.

The PMMU allocates the atomic pages needed for the requested space. TheEnableVector set of bits used in the allocation of atomic pages forincoming packets is a don't care for the GetSpace operation. In otherwords, as long as sufficient consecutive non-allocated atomic pagesexist in a particular block to cover size bytes, the GetSpace(size)operation will succeed even if all the virtual pages in that block aredisabled. Moreover, among non-software-owned blocks, a GetSpaceoperation will be served first using a block that has all its virtualpages disabled. If more than such a block exists, the smallest blocknumber is chosen. If size is 0, GetSpace(size) returns ‘0’.

The second operation software can perform on the PMMU isFreeSpace(packetPage). In this operation the PMU de-allocates atomicpages that were previously allocated (starting at packetPage). Thisspace might have been either automatically allocated by the PMMU as aresult of an incoming packet, or as a result of a GetSpace command.FreeSpace does not return any result to the software. A FreeSpaceoperation on a block with its SoftwareOwned bit asserted is disregarded(nothing is done and no result will be provided to the SPU).

Local Packet Memory

Local Packet Memory (LPM), illustrated as element 219 in FIGS. 2 and 7,has in the instant embodiment a size of 256 KB, 16-byte line width withbyte enables, 2 banks (even/odd), one Read and one Write port per bank,is fully pipelined, and has one cycle latency

The LPM in packet processing receives read and write requests from boththe PMMU and the SIU. An LPM controller guarantees that requests fromthe PMMU have the highest priority. The PMMU reads at most one packetwhile writing another one. The LPM controller guarantees that the PMMUwill always have dedicated ports to the LPM.

Malicious software could read/write the same data that is beingwritten/read by the PMMU. Thus, there is no guarantee that the read andwrite accesses in the same cycle are performed to different 16-byte lineaddresses.

A request to the LPM is defined in this example as a single access(either read or write) of 16-bytes. The SIU generates several requestsfor a masked load or store, which are new instructions known to theinventors and the subject of at least one separate patent application.Therefore, a masked load/store operation can be stalled in the middle ofthese multiple requests if the highest priority PMMU access needs thesame port.

When the PMMU reads or writes, the byte enable signals are assumed to beset (i.e. all 16 bytes in the line are either read or written). When theSIU drives the reads or writes, the byte enable signals are meaningfuland are provided by the SIU.

When the SPU reads a single byte/word in the LPM, the SIU reads thecorresponding 16-byte line and performs the extraction and rightalignment of the desired byte/word. When the SPU writes a singlebyte/word, the SIU generates a 16-byte line with the byte/word in thecorrect location, plus the valid bytes signals.

Prioritization Among Operations

The PMMU may receive up to three requests from three different sources(IB, QS and software) to perform operations. For example, requests maycome from the IB and/or Software: to perform a search for a contiguouschunk of space, to allocate the corresponding atomic page sizes and toprovide the generated packetPage. Requests may also come from the QSand/or Software to perform the de-allocation of the atomic pagesassociated to a given packetPage.

It is required that the first of these operations takes no more than 2cycles, and the second no more than one. The PMMU executes only oneoperation at a time. From highest to lowest, the PMMU block will givepriority to requests from: IB, QS and Software.

Early Full-PMMU Detection

The PMU implements a mechanism to aid in flow control between anyexternal device and the XCaliber processor. Part of this mechanism is todetect that the LPM is becoming full and, in this case, aNoMorePagesOfXsizeInt interrupt is generated to the SPU. The EPM issoftware controlled and, therefore, its state is not maintained by thePMMU hardware.

The software can enable the NoMorePagesOfXsizeInt interrupt byspecifying a virtual page size s. Whenever the PMMU detects that no moreavailable virtual pages of that size are available (i.e. FitsVector[s]is de-asserted for all the blocks), the interrupt is generated. Thelarger the virtual page size selected, the sooner the interrupt will begenerated. The size of the virtual page will be indicated with a 4-bitvalue (0:256 bytes, 1:512 bytes, . . . , 8:64 KB) in an on-the-flyconfiguration register IntIfNoMoreThanXsizePages. When this value isgreater than 8, the interrupt is never generated.

If the smallest virtual page size is selected (256 bytes), theNoMorePagesOfXsizeInt interrupt is generated when the LPM is completelyfull (i.e. no more packets are accepted, not even a 1-byte packet).

In general, if the IntIfNoMoreThanXsizePages is X, the soonest theinterrupt will be generated is when the local packet memory is(100/2^(X))% full. Note that, because of the atomic pages being 256bytes, the LPM could become full with only 3 K-bytes of packet data (3byte per packet, each packet using an atomic page).

Packet Size Mismatch

The PMMU keeps track of how many bytes are being uploaded into the LPMor EPM. If this size is different from the size specified in the firsttwo bytes, a PacketErrorInt interrupt is generated to the SPU. In thiscase the packet with the mismatch packet size is dropped (the alreadyallocated atomic pages will be de-allocated and no packetPage will becreated). No AutomaticDropInt interrupt is generated in this case. Ifthe actual size is more than the size specified in the first two bytes,the remaining packet data being received from the ASIC is gracefullydiscarded.

When a packet size mismatch is detected on an inbound device identifierD (D=0,1), the following packets received from that same deviceidentifier are dropped until software writes (any value) into aClearErrorD configuration register.

Bus Error Recovering

Faulty packet data can arrive to or leave the PMU due to external buserrors. In particular the network input interface may notify that the16-byte chunk of data sent in has a bus error, or the SIU may notifythat the 16-byte chunk of data downloaded from EPM has a bus error. Inboth cases, the PMMU generates the PacketErrorInt interrupt to notifythe SPU about this event. No other information is provided to the SPU.

Note that if an error is generated within the LPM, it will not bedetected since no error detection mechanism is implemented in thison-chip memory. Whenever a bus error arises, no more data of theaffected packet will be received by the PMU. This is done by the SIU inboth cases. For the first case the PMMU needs to de-allocate the alreadyallocated atomic pages used for the packet data received previous to theerror event.

When a bus error is detected on an inbound device identifier D (D=0,1),the following packets received from that same device identifier aredropped until software writes (any value) into a ClearErrorD (D=0,1)configuration register.

Queuing System (QS)

The queueing system (QS) in the PMU of the XCaliber processor hasfunctions of holding packet identifiers and the state of the packetscurrently in-process in the XCaliber processor, keeping packets sortedby their default or software-provided priority, selecting the packetsthat need to be pre-loaded (in the background) into one of the availablecontexts, and selecting those processed packets that are ready to besent out to an external device.

FIG. 9 is a block diagram showing the high-level communication betweenthe QS and other blocks in the PMU and SPU. When the PMMU creates apacketPage, it is sent to the QS along with a queue number and thedevice identifier. The QS enqueues that packetPage in the correspondingqueue and associates a number (packetNumber) to that packet. Eventually,the packet is selected and provided to the RTU, which loads thepacketPage, packetNumber and selected fields of the packet header intoan available context. Eventually the SPU processes that context andcommunicates to the PMU, among other information, when the processing ofthe packet is completed or the packet has been dropped. For thiscommunication, the SPU provides the packetNumber as the packetidentifier. The QS marks that packet as completed (in the first case)and the packet is eventually selected for downloading from packetmemory.

It is a requirement in the instant embodiment (and highly desirable)that packets of the same flow (same source and destination) need to besent out to the external device in the same order as they arrived to theXCaliber processor (unless software explicitly breaks this ordering).When the SPU begins to process a packet the flow is not known. Keepingtrack of the ordering of packets within a flow is a costly task becauseof the amount of processing needed and because the number of activeflows can be very large, depending on the application. Thus, the orderwithin a flow is usually kept track by using aggregated-flow queues. Inan aggregated-flow queue, packet identifiers from different flows aretreated as from the same flow for ordering purposes.

The QS offloads the costly task of maintaining aggregated-flow queues bydoing it in hardware and in the background. Up to 32 aggregated-flowqueues can be maintained in the current embodiment, and each of thesequeues has an implicit priority. Software can enqueue a packetPage inany of the up to 32 queues, and can move a packetPage identifier fromone queue to another (for example, when the priority of that packet isdiscovered by the software). It is expected that software, if needed,will enqueue all the packetPage identifiers of the packets that belongto the same flow into the same queue. Otherwise, a drop in theperformance of the network might occur, since packets will be sent outof order within the same flow. Without software intervention, the QSguarantees the per-flow order of arrival.

Generic Queue

The QS implements a set of up to 32 FIFO-like queues, which arenumbered, in the case of 32 queues, from 0 to 31. Each queue can have upto 256 entries. The addition of all the entries of all the queues,however, cannot exceed 256. Thus, queue sizes are dynamic. A queue entrycorresponds to a packetPage identifier plus some other information. Upto 256 packets are therefore allowed to be in process at any given timein the XCaliber processor. This maximum number is not visible tosoftware.

Whenever the QS enqueues a packetPage, a number (packetNumber) from 0 to255 is assigned to the packetPage. This number is provided to thesoftware along with the packetPage value. When the software wants toperform an operation on the QS, it provides the packetNumber identifier.This identifier is used by the QS to locate the packetPage (and otherinformation associated to the corresponding packet) in and among itsqueues.

Software is aware that the maximum number of queues in the XCaliberprocessor is 32. Queues are disabled unless used. That is, the softwaredoes not need to decide how many queues it needs up front. A queuebecomes enabled when at least one packet is in residence in that queue.

Several packet identifiers from different queues can become candidatesfor a particular operation to be performed. Therefore, someprioritization mechanism must exist to select the packet identifier towhich an operation will be applied first. Software can configure(on-the-fly) the relative priority among the queues using an“on-the-fly” configuration register PriorityClusters. This is a 3-bitvalue that specifies how the different queues are grouped in clusters.Each cluster has associated a priority (the higher the cluster number,the higher the priority). The six different modes in the instantembodiment into which the QS can be configured are shown in the table ofFIG. 10.

The first column of FIG. 10 is the value in the “on-the-fly”configuration register PriorityClusters. Software controls this number,which defines the QS configuration. For example, for PriorityClusters=2,the QS is configured into four clusters, with eight queues per cluster.The first of the four clusters will have queues 0 through 7, the secondcluster will have queues 8-15, the third clusters 16 through 23, and thelast of the four clusters has queues 24 through 31.

Queues within a cluster are treated fairly in a round robin fashion.Clusters are treated in a strict priority fashion. Thus, the only modethat guarantees no starvation of any queue is when PriorityClusters is0, meaning one cluster of 32 queues.

Inserting a packetPage/deviceId into the QS

FIG. 11 is a diagram illustrating the generic architecture of QS 211 ofFIGS. 2 and 7 in the instant embodiment. Insertion of packetPages andDeviceId information is shown as arrows directed toward the individualqueues (in this case 32 queues). The information may be inserted fromthree possible sources, these being the PMMU, the SPU and re-insertionfrom the QS. There exists priority logic, illustrated by functionelement 1101, for the case in which two or more sources have apacketPage ready to be inserted into the QS. In the instant embodimentthe priority is, in descending priority order, the PMMU, the QS, and theSPU (software).

Regarding insertion of packets from the SPU (software), the software cancreate packets on its own. To do so, it first requests a consecutivechunk of free space of a given size (see the SPU documentation) from thePMU, and the PMU returns a packetPage in case the space is found. Thesoftware needs to explicitly insert that packetPage for the packet to beeventually sent out. When the QS inserts this packetPage, thepacketNumber created is sent to the SPU. Software requests an insertionthrough the Command Unit (see FIG. 2).

In the case of insertion from the QS, an entry residing at the head of aqueue may be moved to the tail of another queue. This operation is shownas selection function 1103.

In the case of insertion from the PMU, when a packet arrives to theXCaliber processor, the PMMU assigns a packetPage to the packet, whichis sent to the QS as soon as the corresponding packet is safely storedin packet memory.

An exemplary entry in a queue is illustrated as element 1105, and hasthe following fields: Valid (1) validates the entry. PacketPage (16) isthe first atomic page number in memory used by the packet. NextQueue (5)may be different from the queue number the entry currently belongs to,and if so, this number indicates the queue into which the packetPageneeds to be inserted next when the entry reaches the head of the queue.Delta (10) contains the number of bytes that the header of the packethas either grown or shrunk. This value is coded in 2's complement.Completed (1) is a single bit that indicates whether software hasfinished the processing of the corresponding packet. DeviceId (2) is thedevice identifier associated to the packet. Before a Complete operationis performed on the packet (described below) the DeviceId field containsthe device identifier of the external device that sent the packet in.After the Complete operation, this field contains the device identifierof the device to which the packet will be sent. Active (1) is a singlebit that indicates whether the associated packet is currently beingprocessed by the SPU. CRCtype (2) indicates to the network outputinterface which type of CRC, if any, needs to be computed for thepacket. Before the Complete operation is performed on the packet, thisfield is 0. KeepSpace (1) specifies whether the atomic pages that thepacket occupies in the LPM will be de-allocated (KeepSpace de-asserted)by the PMMU or not (KeepSpace asserted). If the packet resides in EPMthis bit is disregarded by the PMMU.

The QS needs to know the number of the queue to which the packetPagewill be inserted. When software inserts the packetPage, the queue numberis explicitly provided by an XStream packet instruction, which is afunction of the SPU, described elsewhere in this specification. If thepacketPage is inserted by the QS itself, the queue number is the valueof the NextQueue field of the entry where the packetPage resides.

When a packetPage is inserted by the PMMU, the queue number depends onhow the software has configured (at boot time) the Log2InputQueuesconfiguration register. If Log2InputQueues is set to 0, all thepacketPages for the incoming packets will be enqueued in the same queue,which is specified by the on-the-fly configuration registerFirstInputQueue. If Log2InputQueues is set to k(1<=k<=5), then the k MSBbits of the 3rd byte of the packet determine the queue number. Thus anexternal device (or the network input interface block of the SIU) canassign up to 256 priorities for each of the packets sent into the PMU.The QS maps those 256 priorities into 2^(k), and uses queue numbersFirstInputQueue to FirstInputQueue+2^(k)-1 to insert the packetPages anddeviceId information of the incoming packets.

It is expected that an external device will send the same 5 MSB bits inthe 3^(rd) byte for all packets in the same flow. Otherwise, a drop inthe performance of the network might occur, since packets may be sentback to the external device out-of-order within the same flow. Softwareis aware of whether or not the external device (or SIU) can provide theinformation of the priority of the packet in the 3^(rd) byte.

When packetPage p is inserted into queue q, the PacketPage field of theentry to be used is set to p and the Valid field to ‘1’. The value forthe other fields depend on the source of the insertion. If the source issoftware (SPU), Completed is ‘0’; NextQueue is provided by SPU; DeviceIdis ‘0’; Active is ‘1’; CRCtype is 0; KeepSpace is 0, and Probed is 0.

If the source is the QS, the remaining fields are assigned the valuethey have in the entry in which the to-be-inserted packetPage currentlyresides. If the source is the PMMU, Completed is ‘0’, NextQueue is q,DeviceId is the device identifier of the external device that sent thepacket into XCaliber, Active is ‘0’, CRCtype is 0, KeepSpace is 0, andProbed is 0.

Monitoring Logic

The QS monitors entries into all of the queues to detect certainconditions and to perform the corresponding operation, such as tore-enqueue an entry, to send a packetPage plus some other information)to the PMMU for downloading, or to send a packetPage (plus some otherinformation) to the RTU.

All detections take place in a single cycle and they are done inparallel.

Re-Enqueuing an Entry

The QS monitors all the head entities of the queues to determine whethera packet needs to be moved to another queue. Candidate entries to bere-enqueued need to be valid, be at the head of a queue, and have theNextQueue field value different from the queue number of the queue inwhich the packet currently resides.

If more than one candidate exists for re-enqueueing, the chosen entrywill be selected following a priority scheme described later in thisspecification.

Sending an Entry to the PMMU for Downloading

The QS monitors all the head entities of the queues to determine whethera packet needs to be downloaded from the packet memory. This operationis 1102 in FIG. 11. The candidate entries to be sent out of XCaliberneed to be valid, be at the head of the queue, have the NextQueue fieldvalue the same as the queue number of the queue in which the packetcurrently resides, and have the Completed flag asserted and the Activeflag de-asserted. Moreover the QS needs to guarantee that no pendingreads or writes exist from the same context that has issued the downloadcommand to the QS.

If more than one candidate exists for downloading, the chosen entry willbe selected following a priority scheme described later in thisspecification.

A selected candidate will only be sent to the PMMU if the PMMU requestedit. If the candidate was requested, the selected packetPage, along withthe cluster number from which it is extracted, the tail growth/shrink,the outbound device identifier bit, the CRCtype and the KeepSpace bitsare sent to the PMMU.

FIG. 12 is a table indicating coding of the Deviceid field. If theDeviceid field is 0, then the Outbound Device Identifier is the same asthe Inbound Device Identifier, and so on as per the table.

When an entry is sent to the PMMU, the entry is marked as “beingtransmitted” and it is extracted from the queuing system (so that itdoes not block other packets that are ready to be transmitted and go toa different outbound device identifier). However, the entry is notinvalidated until the PMMU notifies that the corresponding packet hasbeen completely downloaded. Thus, probe-type operations on this entrywill be treated as valid, i.e. as still residing in the XCaliberprocessor.

Reincarnation Effect

As described above, the QS assigns a packetNumber from 0 to 255 (256numbers in total) to each packet that comes into XCaliber and isinserted into a queue. This is done by maintaining a table of 256entries into which packet identifiers are inserted. At this time theValid bit in the packet identifier is also asserted. Because the overallnumbers of packets dealt with by XCaliber far exceeds 256, packetnumbers, of course, have to be reused throughout the running of theXCaliber processor. Therefore, when packets are selected fordownloading, at some point the packetNumber is no longer associated witha valid packet in process, and the number may be reused.

As long as a packet is valid in XCaliber it is associated with thepacketNumber originally assigned. The usual way in which a packetNumberbecomes available to be reused is that a packet is sent by the QS to theRTU for preloading in a context prior to processing. Then when thepacket is fully processed and fully downloaded from memory, the packetidentifier in the table associating packetNumbers is marked Invalid bymanipulating the Valid bit (see FIG. 11 and the text accompanying).

In usual operation the system thus far described is perfectly adequate.It has been discovered by the inventors, however, that there are somesituations in which the Active and Valid bits are not sufficient toavoid contention between streams. One of these situations has to do witha clean-up process, sometimes termed garbage collection, in whichsoftware monitors all packet numbers to determine when packets haveremained in the system too long, and discards packets under certainconditions, freeing space in the system for newly-arriving packets.

In these special operations, like garbage collection, a stream must gainownership of a packet, and assure that the operation it is to perform onthe packet actually gets performed on the correct packet. As softwareprobes packets, however, and before action may be taken, because thereare several streams operating, and because the normal operation of thesystem may also send packets to the RTU, for example, it is perfectlypossible in these special operations that a packet probed may beselected and effected by another stream before the special operation iscompleted. A packet, for example, may be sent to the RTU, processed, anddownloaded, and a new packet may then be assigned to the packetNumber,and the new packet may even be stored at exactly the same packetPage asthe original packet. There is a danger, then, that the specialoperations, such as discarding a packet in the garbage collectionprocess, may discard a new and perfectly valid packet, instead of thepacket originally selected to be discarded. This, of course, is just oneof potentially many such special operations that might lead to trouble.

Considering the above, the inventors have provided a mechanism forassuring that, given two different absolute points in time, time s andtime r, for example, that a valid packetNumber at time s and the samepacketNumber at time r, still is associated to the same packet. A simpleprobe operation is not enough, because at some time after s and beforetime r the associated packet may be downloaded, and another (anddifferent) packet may have arrived, been stored in exactly the samememory location as the previous packet, and been assigned the samepacketNumber as the downloaded packet.

The mechanism implemented in XCaliber to ensure packetNumber associationwith a specific packet at different times includes a probe bit in thepacket identifier. When a first stream, performing a process such asgarbage collection, probes a packet, a special command, called Probe&Setis used. Probe&Set sets (asserts) the probe bit, and the usualinformation is returned, such as the value for the Valid bit, the Activebit, the packetPage address, and the old value of the probe bit. Thefirst stream then executes a Conditional Activate instruction, describedelsewhere in this specification, to gain ownership of the packet. Also,when the queuing system executes this Conditional Activate instructionit asserts the active bit of the packet. Now, at any time after theprobe bit is set by the first stream, when a second stream at a latertime probes the same packet, the asserted probe bit indicates that thefirst stream intends to gain control of this packet. The second streamnow knows to leave this packet alone. This probe bit is de-asserted whena packet enters the XCaliber processor and a new (non-valid) number isassigned.

Sending an Entry to the RTU

The RTU uploads in the SPU background to the XCaliber processor somefields of the headers of packets that have arrived, and have beencompletely stored into packet memory. This uploading of the header of apacket in the background may occur multiple times for the same packet.The QS keeps track of which packets need to be sent to the RTU. Theselection operation is illustrated in FIG. 11 as 1104.

Whenever the RTU has chosen a context to pre-load a packet, it notifiesthe QS that the corresponding packet is no longer an inactive packet.The QS then marks the packet as active.

Candidate entries to be sent to the RTU need to be valid, to be theoldest entry with the Active and Completed bits de-asserted, to have theNextQueue field value the same as the queue number of the queue in whichthe packet currently resides, and to conform to a limitation that nomore than a certain number of packets in the queue in which thecandidate resides are currently being processed in the SPU. More detailregarding this limitation is provided later in this specification. Whenan entry is sent to the RTU for pre-loading, the corresponding Activebit is asserted.

A queue can have entries with packet identifiers that already have beenpresented to the RTU and entries that still have not. Every queue has apointer (NextPacketForRTU) that points to the oldest entry within thatqueue that needs to be sent to the RTU. Within a queue, packetidentifiers are sent to the RTU in the same order they were inserted inthe queue.

The candidate packet identifiers to be sent to the RTU are those pointedto by the different NextPacketForRTU pointers associated with thequeues. However, some of these pointers might point to a non-existententry (for example, when the queue is empty or when all the entries havealready been sent to the RTU). The hardware that keeps track of thestate of each of the queues determines these conditions. Besides being avalid entry pointed to by a NextPacketForRTU pointer, the candidateentry needs to have associated with it an RTU priority (described laterin this specification) currently not being used by another entry in theRTU. If more than a single candidate exists, the chosen entry isselected following a priority scheme described later in thisspecification.

As opposed to the case in which an entry is sent to the PMMU fordownloading, an entry sent to the RTU is not extracted from its queue.Instead, the corresponding NextPacketForRTU pointer is updated, and thecorresponding Active bit is asserted.

The QS sends entries to an 8-entry table in the RTU block as long as theentry is a valid candidate and the corresponding slot in the RTU tableis empty. The RTU will accept, at most, 8 entries, one per eachinterrupt that the RTU may generate to the SPU.

The QS maps the priority of the entry (given by the queue number whereit resides) that it wants to send to the RTU into one of the 8priorities handled by the RTU (RTU priorities). This mapping is shown inthe table of FIG. 13, and it depends on the number of clusters intowhich the different queues are grouped (configuration registerPriorityClusters) and the queue number in which the entry resides.

The RTU has a table of 8 entries, one for each RTU priority. Every entrycontains a packet identifier (packetPage, packetNumber, queue#) and aValid bit that validates it. The RTU always accepts a packet identifierof RTU priority p if the corresponding Valid bit in entry p of thattable is de-asserted. When the RTU receives a packet identifier of RTUpriority p from the QS, the Valid bit of entry p in the table isasserted, and the packet identifier is stored. At that time the QS canupdate the corresponding NextPacketForRTU pointer.

Limiting the Packets Sent within a Queue

Software can limit the number of packets that can be active (i.e. beingprocessed by any of the streams in the SPU) on a per-queue basis. Thisis achieved through a MaxActivePackets on-the-fly configurationregister, which specifies, for each queue, a value between 1 and 256that corresponds to the maximum number of packets, within that queue,that can be being processed by any stream.

The QS maintains a counter for each queue q which keeps track of thecurrent number of packets active for queue q. This counter isincremented whenever a packet identifier is sent from queue q to theRTU, a Move operation moves a packet into queue q, or an Insertoperation inserts a packet identifier into queue q; and decremented whenany one the following operations are performed in any valid entry inqueue q: a Complete operation, an Extract operation, a Move operationthat moves the entry to a different queue, or a MoveAndReactivateoperation that moves the entry to any queue (even to the same queue).Move, MoveAndReactivate, Insert, Complete and Extract are operationsdescribed elsewhere in this specification.

Whenever the value of the counter for queue q is equal to or greaterthan the corresponding maximum value specified in the MaxActivePacketsconfiguration register, no entry from queue q is allowed to be sent tothe RTU. The value of the counter could be greater since software canchange the MaxActivePackets configuration register for a queue to avalue lower than the counter value at the time of the change, and aqueue can receive a burst of moves and inserts.

Software Operations on the QS

Software executes several instructions that affect the QS. The followingis a list of all operations that can be generated to the QS as a resultof the dispatch by the SPU core of an XStream packet instruction:

Insert(p,q): the packetPage p is inserted into queue q. A ‘1’ will bereturned to the SPU if the insertion was successful, and a ‘0’ if not.The insertion will be unsuccessful only when no entries are available(i.e. when all the 256 entries are valid).

Move(n,q): asserts to q the NextQueue field of the entry in whichpacketNumber n resides.

MoveAndReactivate(n,q): asserts to q the NextQueue field of the entry inwhich packetNumber n resides; de-asserts the Active bit.

Complete(n,d,e): asserts the Completed flag, the Delta field to d andthe deviceId field to e of the entry in which packetNumber n resides.De-asserts the Active bit and de-asserts the KeepSpace bit.

CompleteAndKeepSpace(n,d,e): same as Complete( ) but it asserts theKeepSpace bit.

Extract(n): resets the Valid flag of the entry in which packetNumber nresides.

Replace(n,p): the PacketPage field of the entry in which packetNumber nresides is set to packetPage p.

Probe(n): the information whether the packetNumber n exists in the QS ornot is returned to the software. In case it exists, it returns thePacketPage, Completed, NextQueue, DeviceId, CRCtype, Active, KeepSpaceand Probed fields.

ConditionalActivate(n): returns a ‘1’ if the packetNumber n is valid,Probed is asserted, Active is de-asserted, and the packet is not beingtransmitted. In this case, the Active bit is asserted.

The QS queries the RTU to determine whether the packet identifier of thepacket to be potentially activated is in the RTU table, waiting to bepreloaded, or being preloaded. If the packet identifier is in the table,the RTU invalidates it. If the query happens simultaneously with thestart of preloading of that packet, the QS does not activate the packet.

ProbeAndSet(n): same as Probe( ) but it asserts the Probed bit (thereturned Probed bit is the old Probed bit).

Probe(q): provides the size (i.e. number of valid entries) in queue q.

A Move( ), MoveAndReactivate( ), Complete( ), CompleteAndKeepSpace( ),Extract( ) and Replace( ) on an invalid (i.e. non-existing) packetNumberis disregarded (no interrupt is generated).

A Move, MoveAndReactivate, Complete, CompleteAndKeepSpace, Extract andReplace on a valid packetNumber with the Active bit de-asserted shouldnot happen (guaranteed by software). If it happens, results areundefined. Only the Insert, Probe, ProbeAndSet and ConditionalActivateoperations reply back to the SPU.

If software issues two move-like operations to the PMU that affect thesame packet, results are undefined, since there is no guarantee that themoves will happen as software specified.

FIG. 14 is a table showing allowed combinations of Active, Completed,and Probed bits for a valid packet.

Basic Operations

To support the software operations and the monitoring logic, the QSimplements the following basic operations:

-   1. Enqueue an entry at the tail of a queue.-   2. Dequeue an entry from the queue in which it resides.-   3. Move an entry from the head of the queue wherein it currently    resides to the tail of another queue.-   4. Provide an entry of a queue to the RTU.-   5. Provide the size of a queue.-   6. Update any of the fields associated to packetNumber.

Operations 1, 2, 4 and 6 above (applied to different packets at the sametime) are completed in 4 cycles in a preferred embodiment of the presentinvention. This implies a throughput of one operation per cycle.

Some prioritization is necessary when two or more operations could startto be executed at the same time. From highest to lowest priority, theseevents are inserting from the PMMU, dequeuing an entry, moving an entryfrom one queue to another queue, sending an entry to the RTU forpre-loading, or a software operation. The prioritization among thesoftware operations is provided by design since software operations arealways executed in order.

Early QS Full Detection

The PMU implements a mechanism to aid in flow control between the ASIC(see element 203 in FIG. 2) and the XCaliber processor. Part of thismechanism is to detect that the QS is becoming full and, in this case, aLessThanXpacketIdEntriesInt interrupt is generated to the SPU. Thesoftware can enable this interrupt by specifying (in aIntIfLessThanXpacketIdEntries configuration register) a number z largerthan 0. An interrupt is generated when 256−y<z, being y the total numberof packets currently in process in XCaliber. When z=0, the interruptwill never occur.

Register Transfer Unit (RTU)

A goal of the RTU block is to pre-load an available context withinformation of packets alive in XCaliber. This information is thepacketPage and packetNumber of the packet and some fields of its header.The selected context is owned by the PMU at the time of the pre-loading,and released to the SPU as soon as it has been pre-loaded. Thus, the SPUdoes not need to perform the costly load operations to load the headerinformation and, therefore, the overall latency of processing packets isreduced.

The RTU receives from the QS a packet identifier (packetPage,packetNumber) and the number of the queue from which the packet comesfrom) from the QS. This identifier is created partly by the PMMU as aresult of a new packet arriving to XCaliber through the network inputinterface (packetPage), and partly by the QS when the packetPage anddevice identifier are enqueued (packetNumber).

Another function of the RTU is to execute masked load/store instructionsdispatched by the SPU core since the logic to execute a maskedload/store instruction is similar to the logic to perform a pre-load.Therefore, the hardware can be shared for both operations. For thisreason, the RTU performs either a masked load/store or a pre-load, butnot both, at a time. The masked load/store instructions arrive to theRTU through the command queue (CU) block.

Context States

A context can be in one of two states: PMU-owned or SPU-owned. Theownership of a context changes when the current owner releases thecontext. The PMU releases a context to the SPU in three cases. Firstly,when the RTU has finished pre-loading the information of the packet intothe context. Secondly, the PMU releases a context to the SPU when theSPU requests a context to the RTU. In this case, the RTU will release acontext if it has one available for releasing. Thirdly, all eightcontexts are PMU-owned. Note that a context being pre-loaded isconsidered to be a PMU-owned context.

The SPU releases a context to the RTU when the SPU dispatches an XStreamRELEASE instruction.

Pre-Loading a Context

At boot time, the PMU owns 7 out of the 8 contexts that are available inthe embodiment of the invention described in the present example, andthe SPU owns one context. The PMU can only pre-load information of apacket to a context that it owns. The process of pre-loading informationof a packet into a context is divided into two steps. A first phase toload the address (the offset within the packet memory address space),from where the packet starts. This offset points to the first byte ofthe two-byte value that codes the size in bytes of the packet. In thecase that the packet has been time stamped and HeaderGrowthOffset is not0, the time stamp value is located at offset-4. The offset address iscomputed as (packetPage<<8)|(HeaderGrowthOffset<<4). This offset isloaded into register number StartLoadingRegister in the selectedcontext. StartLoadingRegister is a boot-time configuration register. ThepacketNumber value is loaded in register number StartLoadingRegister+1.

The second phase is to load the packet header. The packet header isloaded using registers StartLoadingRegister+2, StartLoadingRegister+3, .. . (as many as needed, and as long as there exist GPR registers). ThePatternMatchingTable[q] (q being the queue number associated to thepacket) mask specifies how the header of the packet will be loaded intothe GPR registers of the context. The PatternMatchingTable is anon-the-fly configuration register that contains masks. To obtain theheader data, the RTU requests the SIU to read as many 16-byte lines ofpacket data as needed into the packet memory. The RTU, upon receivingthe 16-byte lines from packet memory (either local or external), selectsthe desired bytes to load into the context using pattern mask to controlthis operation.

The step described immediately above of loading the packet header may bedisabled by software on a per-queue basis through the on-the-flyPreloadMaskNumber configuration register. This register specifies, foreach of the 32 possible queues in the QS, which mask (from 0 to 23) inthe PatternMatchingTable is going to be used for the pre-loading. If avalue between 24 and 31 is specified in the configuration register, itis interpreted by the RTU as not to perform.

The RTU only loads the GPR registers of a context. The required CP0registers are initialized by the SPU. Since the context loaded is aPMU-owned context, the RTU has all the available write ports to thatcontext (4 in this embodiment) to perform the loading.

Whenever the pre-loading operation starts, the RTU notifies this eventto the SPU through a dedicated interface. Similarly, when thepre-loading operation is completed, the RTU also notified the SPU. Thusthe SPU expects two notifications (start and end) for each packetpre-load. A special notification is provided to the SPU when the RTUstarts and ends a pre-load in the same cycle (which occurs when the stepof loading packet header is disabled). In all three cases, the RTUprovides the context number and the contents of the CodeEntryPointconfiguration register associated to the packet. In the case that thePMU releases a context to the SPU because all eight contexts arePMU-owned, the contents of the CodeEntryPointSpecial are provided to theSPU. The RTU has an 8-entry table (one for each context), each entryhaving a packet identifier ready to be pre-loaded and a valid bit thatvalidates the entry. The RTU selects always the valid identifier of thehighest entry index to do the pre-load. When a context is associated tothis identifier, the corresponding valid bit is de-asserted. The RTUpre-loads one context at a time. After loading a context, the context isreleased to the SPU and becomes a SPU-owned context. At this point theRTU searches its table for the next packet to be pre-loaded into acontext (in case there is at leas one PMU-owned context).

Pattern-Matching Table

FIG. 15 illustrates a Pattern Matching Table which is an on-the-flyconfiguration register that contains a set of sub-masks. The RTU can useany sub-mask (from 0 to 23) within this table for a pre-loading acontext. Sub-masks can also be grouped into a larger mask containing twoor more submasks.

FIG. 16 illustrates the format of a mask. A mask is a variable number (1to 8) of sub-masks of 32×2 bits each, as shown. Every sub-mask has anassociated bit (EndOfMask) that indicates whether the composite maskfinishes with the corresponding sub-mask, or it continues with the nextsub-mask. The maximum total number of sub-masks is 32, out of which 24(sub-mask indexes 0 to 23) are global, which means any stream in the SPUcan use and update them, and 8 are per-stream sub-masks. The per-streamsub-masks do not have an EndOfMask bit, which is because no grouping ofper-stream sub-masks is allowed.

The two 32-bit vectors in each sub-mask are named SelectVector andRegisterVector. The SelectVector indicates which bytes from the headerof the packet will be stored into the context. The RegisterVectorindicates when to switch to the next consecutive register within thecontext to keep storing the selected bytes by the SelectVector. Thebytes are always right aligned in the register.

FIG. 17 shows an example of a pre-load operation using the mask in FIG.16. A bit asserted in the SelectVector indicates that the correspondingbyte of the header are stored into a register. In the example, bytes 0,1 and 7 of the header are loaded into GPR number StartLoadingRegister+2in bytes 0, 1 and 2, respectively (i.e. the header bytes areright-aligned when loaded into the register). A bit asserted in theRegisterVector indicates that no more header bytes are loaded into thecurrent GPR register, and that the next header bytes, if any, are loadedinto the next (consecutively) GPR register. In the example, bytes 12 and13 of the header are loaded into GPR number StartLoadingRegister+3.

Selecting a PMU-Owned Context

There are a total of eight functional units in the SPU core. However,due to complexity-performance tradeoffs, a stream (context) can onlyissue instructions to a fixed set of 4 functional units.

The RTU may own at any given time several contexts. Therefore, logic isprovided to select one of the contexts when a pre-load is performed, orwhen a context has to be provided to the SPU. This logic is definedbased on how the different streams (contexts) in the SPU core canpotentially dispatch instructions to the different functional units, andthe goal of the logic is to balance operations that the functional unitsin the SPU can potentially receive.

The selection logic takes as inputs eight bits, one per context, thatindicates whether that context is PMU or SPU-owned. The logic outputswhich PMU-owned context(s) that can be selected.

1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 3, 20, 5,6, 7, 24, 9, 10, 11, 12, 13, 14, 15, 32, 33, 34, 3, 36, 5, 6, 7, 40, 9,10, 11, 12, 13, 14, 15, 48, 48, 48, 51, 48, 53, 54, 7, 48, 57, 58, 11,60, 13, 14, 15, 64, 65, 66, 3, 68, 5, 6, 7, 72, 9, 10, 11, 12, 13, 14,15, 80, 80, 80, 83, 80, 85, 86, 7, 80, 89, 90, 11, 92, 13, 14, 15, 96,96, 96, 99, 96, 101, 102, 7, 96, 105, 106, 11, 108, 13, 14, 15, 112,112, 112, 112, 112, 112, 112, 119, 112, 112, 112, 123, 112, 125, 126,15, 128, 129, 130, 3, 132, 5, 6, 7, 136, 9, 10, 11, 12, 13, 14, 15, 144,144, 144, 147, 144, 149, 150, 7, 144, 153, 154, 11, 156, 13, 14, 15,160, 160, 160, 163, 160, 165, 166, 7, 160, 169, 170, 11, 172, 13, 14,15, 176, 176, 176, 176, 176, 176, 176, 183, 176, 176, 176, 187, 176,189, 190, 15, 192, 192, 192, 195, 192, 197, 198, 7, 192, 201, 202, 11,204, 13, 14, 15, 208, 208, 208, 208, 208, 208, 208, 215, 208, 208, 208,219, 208, 221, 222, 15, 224, 224, 224, 224, 224, 224, 224, 231, 224,224, 224, 235, 224, 237, 238, 15, 240, 240, 240, 240, 240, 240, 240,240, 240, 240, 240, 240, 240, 240, 240

The selection logic is specified with the previous list of 254 numbers.Each number is associated to a possible combination of SPU/PMU-ownedcontext. For example, the first number corresponds to the combination‘00000001’ i.e. context number 0 is PMU owned and context numbers 1 to 7are SPU owned (LSB digit corresponds to context 0, MSB digit to context7; digit value of 0 means SPU owned, digit value of 1 means PMU owned).The second number corresponds to combination ‘00000010’, the third tocombination ‘00000011’, and so forth up to combination ‘11111110’. The19^(th) combination (‘00010011’) has associated number 3 (or ‘00000011’)in the previous list, which means that context 0 and 1 can be selected.Context 4 could also be selected, however it is not the best choice tobalance the use of the functional units in the SPU core.

Interrupt when No Context is Available

The RTU has a table of 8 entries named NewPacketIdTable). Entry p inthis table contains a packet identifier (packetPage, packetNumber andqueue number) with an RTU-priority of p, and a Valid bit that validatesthe identifier. When the RTU is not busy pre-loading or executing amasked load/store, it will obtain from this table the valid identifierwith the highest RTU-priority. In case it exists and there is at leastone PMU-owned context, the RTU will start the pre-loading of a PMU-ownedcontext, and it will reset the Valid bit in the table.

In case there is no PMU-owned context, the RTU sits idle (assuming nosoftware operation is pending) until a context is released by the SPU.At that point in time the RTU obtains, again, the highest validRTU-priority identifier from the NewPacketIdTable (since a newidentifier with higher RTU priority could have been sent by the QS whilethe RTU was waiting for a context to be released by the SPU). The Validbit is reset and the packet information starts being pre-loaded into theavailable context. At this point the RTU is able to accept a packet withRTU priority p from the QS.

When an identifier with a RTU priority of p is sent by the QS to theRTU, it is loaded in entry p in the NewPacketIdTable, and the Valid bitis set. At this time, if the number of valid identifiers (withoutcounting the incoming one) in the NewPacketIdTable is equal or largerthan the current available PMU-owned contexts (without counting thecontext that the RTU currently might be loading), then aPacketAvailableButNoContextPriorityP Int interrupt is generated to theSPU. P ranges from 0 to 7, and its value is determined by a boot-timeconfiguration flag PacketAvailableButNo ContextIntMapping. If this flagis ‘0’, P is determined by the 3-bit boot-time configuration registerDefaultPacketPriority. If this flag is ‘1’, P is the RTU priority.However, the PacketAvailableButNoContextPriorityPint will not begenerated if the corresponding configuration flag PacketAvailableButNoContextPriorityPintEnable is de-asserted.

The SPU, upon receiving the interrupt, decides whether or not to releasea context that it owns so that the RTU can pre-load the packetPage,packetNumber and header information of the new packet.

When the RTU generates a PacketAvailableButNoContext PriorityPIntinterrupt, it may receive after a few cycles a context that has beenreleased by the SPU. This context, however, could have been releasedwhen, for example, one of the streams finished the processing of apacket. This can happen before the interrupt service routine for thePacketAvailable ButNoContextPriorityPInt interrupt finishes. Thus, whena context is released due to the ISR completion, the packet pre-loadthat originated the interrupt already might have used the context firstreleased by another stream in the SPU. Thus, the context released due tothe interrupt will be used for another (maybe future) packet pre-load.If no other entry is valid in the NewPacketIdTable, the context is beused and sits still until either an identifier arrives to the RTU or theSPU requesting a context to the RTU.

Whenever a context becomes SPU-owned, and the RTU has a pre-loadpending, the RTU selects the most priority pending pre-load (whichcorresponds to the highest-valid entry in the NewPacketTable), and willstart the preload. If the PacketAvailableButNoContextPriorityintinterrupt associated to this level was asserted, it gets de-assertedwhen the pre-load starts.

Software Operations on the RTU

Software executes a number of instructions that affect the RTU.Following is a list of all operations that can be generated to the RTUas a result of dispatch by the SPU core of an XStream packetinstruction. The operations arrive to the RTU through the command queue(CU), along with the context number associated to the stream that issuedthe instruction:

-   1. Release(c): context number c becomes PMU owned.-   2. GetContext: the RTU returns the number of a PMU-owned context    number. This context, if it exists, becomes SPU owned and a success    flag is returned asserted; otherwise it is return de-asserted, in    which case the context number is meaningless.-   3. MaskedLoad(r,a,m), MaskedStore(r,a,m): the SPU core uses the RTU    as a special functional unit to execute the masked load/store    instructions since the logic to execute a masked load/store    instruction is similar to the logic to perform a pre-load.    Therefore, the hardware can be shared for both operations. For this    reason, the RTU performs either a masked load/store or a pre-load,    but not both at a time. For either the masked load or masked store,    the RTU will receive the following parameters:    -   (a) A mask number m that corresponds to the index of the first        submask in the PatternMatchingTable to be used by the masked        load/store operation.    -   (b) A 36-bit address a that points to the first byte in (any)        memory to which the mask will start to be applied.    -   (c) A register number r (within the context number provided)        that corresponds to the first register involved in the masked        load/store operation. Subsequent registers within the same        context number will be used according to the selected mask.

For masked load/store operations, the mask can start to be applied atany byte of the memory, whereas in a pre-load operation (a masked-loadlike operation) the mask will always be applied starting at a 16-byteboundary address since packet data coming from the network inputinterface is always stored in packet memory starting at the LSB byte ina 16-byte line.

The MaskedLoad, MaskedStore and GetContext operations communicate to theSPU when they complete through a dedicated interface between the RTU andthe SPU. The RTU gives more priority to a software operation than packetpre-loads. Pre-loads access the packet memory whereas the maskedload/store may access any memory in the system as long as it is notcacheable or write-through. If not, results are undefined.

The RTU is able to execute a GetContext or Release command whileexecuting a previous masked load/store command.

Programming Model

Software can configure, either at boot time or on the fly, several ofthe features of the PMU. All of the features configurable at boot timeonly, and some configurable on the fly, must happen only when the SPU isrunning in a single-stream mode. If not, results are undefined. The PMUdoes not check in which mode the SPU is running.

Software can update some of the information that the PMU maintains for agiven packet, and also obtain this information. This is accomplished bysoftware through new XStream packet instructions that are the subject ofseparate patent applications. These instructions create operations ofthree different types (depending on which block of the PMU the operationaffects, whether PMMU, QS or RTU) that will be executed by the PMU. Someof the operations require a result from the PMU to be sent back to theSPU.

The packet memory and configuration space are memory mapped. The SIUmaintains a configuration register (16 MB aligned) with the base addressof the packet memory, and a second configuration register with the baseaddress of EPM. Software sees the packet memory as a contiguous space.The system, however, allows the EPM portion of the packet memory to bemapped in a different space.

The SIU also maintains a third configuration register with the base ofthe PMU configuration register space. All the load/store accesses to LPMand configuration space performed by the SPU reach the PMU through theSIU. The SIU determines to which space the access belongs, and lets thePMU know whether the access is to LPM or to the PMU configuration space.Accesses to the EPM are transparent to the PMU.

The PMU can interrupt the SPU when certain events happen. Software candisable all these interrupts through configuration registers.

Configuration Registers

The configuration registers of the PMU reside in the PMU ConfigurationSpace of the XCaliber address space. The base address of this space ismaintained by the SIU and does not need to be visible by the PMU. TheSIU notifies to the PMU with a signal when a read/write access performedby the SPU belongs to this space, along with the information needed toupdate the particular register on a write access.

Some of the PMU configuration registers can be configured only at boottime, and some can be configured on the fly. All boot-time configurableand some on-the-fly configurable registers need to be accessed insingle-stream mode. A boot-time configurable register should only beupdated if the PMU is in reset mode. Results are undefined otherwise.The PMU will not check whether the SPU is indeed in single-stream modewhen a single-stream mode configuration register is updated. All theconfiguration registers come up with a default value after the resetsequence.

In the instant embodiment 4 KB of the XCaliber address space isallocated for the PMU configuration space. In XCaliber's PMU, some ofthese configuration registers are either not used or are sparsely used(i.e. only some bits of the 32-bit configuration register word aremeaningful). The non-defined bits in the PMU configuration space arereserved for future PMU generations. Software can read or write thesereserved bits but their contents, although fully deterministic, areundefined.

FIG. 18 shows the PMU Configuration Space, which is logically dividedinto 32-bit words. Each word or set of words contains a configurationregister.

FIGS. 19 a and 19 b are two parts of a table showing mapping of thedifferent PMU configuration registers into the different words of theconfiguration space. The block owner of each configuration register isalso shown in the table.

Following is the list of all configuration registers in this particularembodiment along with a description and the default value (after PMUreset). For each of the configuration registers, the bit width is shownin parenthesis. Unless otherwise specified, the value of theconfiguration register is right aligned into the corresponding wordwithin the configuration space.

Boot-Time Only Configuration Registers:

-   1. Log2InputQueues (5)    -   (a) Default Value: 0    -   (b) Description: Number of queues in the QS used as input queues        (i.e. number of queues in which packetPages/deviceIds from the        PMMU will be inserted).-   2. PriorityClustering (3)    -   (a) Default Value: 5 (32 clusters)    -   (b) Description: Specifies how the different queues in the QS        are grouped in priority clusters (0: 1 cluster, 1: 2 clusters,        2: 4 clusters, . . . , 5: 32 clusters).-   3. HeaderGrowthOffset (4)    -   (a) Default Value: 0    -   (b) Description: Number of empty 16-byte chunks that will be        left in front of the packet when it is stored in packet memory.        Maximum value is 15 16-byte chunks. Minimum is 0.-   4. TailGrowthOffset (6)    -   (a) Default Value: 0    -   (b) Description: Number of empty 16-byte chunks that will be        left at the end of the packet when it is stored in packet        memory. Maximum value is 63 16-byte chunks. Minimum is 0.-   5. PacketAvailableButNoContextIntMapping (1)    -   (a) Default Value: 0    -   (b) Description: Specifies the P in the        PacketAvailableButNoContextPriorityPInt interrupt, if enabled.        The possible values are:        -   (1) 0: P is specified by the DefaultPacketPriority register.        -   (2) 1: P is the RTU priority.-   6. StartLoadingRegister (5)    -   (a) Default Value: 1    -   (b) Description: Determines the first GPR register number to be        loaded by the RTU when performing the background load of the        packet header on the chosen context. In this register, the value        (packetPage<<8)|(HeaderGrowthOffset<<4) is loaded. The        packetNumber is loaded in the next GPR register. The following        GPR registers will be used to pre-load the packet header data        following PatternMatchingMask0 mask if this feature is enabled.-   7. PreloadMaskNumber (32×5)    -   (a) Default Value: mask 31 for all queues (i.e. pre-load of        header is disabled).    -   (b) Description: It specifies, for each of the 32 possible        queues in the QS, which mask in the PatternMatchingTable is        going to be used for pre-loading.

FIGS. 19 a-c show a mapping of the PreloadMaskNumber configurationregister.

The configuration registers described above are the boot-time-onlyconfiguration registers in the instant example. Immediately below arelisted the On-The-Fly configuration registers.

Single-Stream Configuration Registers

-   1. OverflowEnable (1)    -   (a) Default Value: 0    -   (b) Description: Enables/disables the overflow of packets in        case they do not fit into LPM. When disabled, these packets are        dropped.-   2. PatternMatchingTable (24×(32×2+1)    -   (a) Default Value (per each of the 24 entries):        -   (1) SelectVector: select all bytes        -   (2) RegisterVector: store 4 consecutive bytes per register        -   (3) EndOfMask: 1    -   (b) Description: It specifies, for masked load/store operations,        which bytes to load/store and in which (consecutive) registers.        Mask 0 of this table is used by the RTU to pre-load, in the        background, some bytes of the header of the packet in one of the        available contexts. There are a total of 24 masks.    -   (c) Note: Mask 0 needs to be written when the PMU is freezed        (see Section 0), otherwise results are undefined.

FIG. 21 illustrates the PatternMatchingTable described immediatelyabove.

-   3. Freeze (1)    -   (a) Default Value: 1    -   (b) Description: Enables/disables the freeze mode.-   4. Reset (1)    -   (a) Default Value: 0    -   (b) Description: When set to 1, forces the PMU to perform the        reset sequence. All packet data in the PMU will be lost. After        the reset sequence all the configuration registers will have the        default values.        Multi-Stream Configuration Registers-   1. ClearErrorD (D=0,1)    -   (a) Default Value: 0    -   (b) Description: When written by software (with any data), the        packet error condition detected on device identifier D is        cleared.-   2. PacketAvailableButNoContextPriorityPintEnable (8) [P=0 . . . 7]    -   (a) Default Value: 0 (for all levels)    -   (b) Description: Enables/disables the        PacketAvailableButNoContextPriorityPint interrupt.-   3. AutomaticPacketDropIntEnable (1)    -   (a) Default Value: 1    -   (b) Description: Enables/disables the AutomaticPacketDropInt        interrupt.-   4. TimeStampEnable (1)    -   (a) Default Value: 0    -   (b) Description: Enables/disables the time stamp of packets.        When enabled and HeaderGrowthOffset is greater than 0, a 4-byte        time stamp is appended to the packet before it is written into        the packet memory.-   5. PacketErrorIntEnable (1)    -   (a) Default Value: 0    -   (b) Description: Enables/disables the PacketErrorInt interrupt.-   6. VirtualPageEnable (9×4)    -   (a) Default Value: all virtual pages enabled for all blocks.    -   (b) Description: Enables/disables the virtual pages for each of        the 4 blocks that the LPM is divided into. There are up to 9        virtual pages, from 256 bytes (enabled by the LSB bit) up to 64K        bytes (enabled by the MSB bit), with all power-of-two sizes in        between.

FIG. 22 illustrates the VirtualPageEnable register.

-   7. OverflowAddress (24)    -   (a) Default Value: 0x40000 (the first atomic page in the EPM)    -   (b) Description: the 16 MSB bits correspond to the atomic page        number in packet memory into which the packet that is overflowed        will start to be stored. The 8 LSB are hardwired to ‘0’ (i.e.        any value set by software to these bits will be disregarded).        OverflowAddress is then the offset address within the 16 MB        packet memory. The SIU will translate this offset into the        corresponding physical address into the EPM. The first 1K atomic        pages of the packet memory correspond to the LPM. If software        sets the 16 MSB of OverflowAddress to 0 . . . 1023, results are        undefined. When a packet is overflowed, the 16 MSB bits of        OverflowAddress become the packetPage for that packet. The SPU        allows the next packet overflow when it writes into this        configuration register.-   8. IntIfNoMoreXsizePages (4)    -   (a) Default Value: 0xF (i.e. the interrupt will never be        generated)    -   (b) Description: Specifies the index of a virtual page (0:256        bytes, 1:512 bytes, . . . , 8:64 K bytes, 9-15: no virtual        page). Whenever the PMMU detects that there are no more virtual        pages of that size in all the LPM, the NoMoreThanXSizePagesInt        interrupt will be generated to the SPU.-   9. IntIfLessThanXpacketIdEntries (9)    -   (a) Default Value: 0    -   (b) Description: Minimum number of entries in the QS available        for new packet identifiers. If the actual number of available        entries is less than this number, an interrupt will be generated        to the SPU. If this number is 0, the LessThanXPacketIdEntriesInt        interrupt will not be generated.-   10. DefaultPacketPriority (3)    -   (a) Default Value: 0    -   (b) Description: Provides the priority level for the        PacketAvailableButNoContextInt interrupt when        PacketAvailableButNoContextMapping is 0.-   11. ContextSpecificPatternMatchingMask: (8×(32×2))    -   (a) Default Value:        -   (1) SelectVector: select all bytes        -   (2) RegisterVector: store 4 bytes in each register            (EndOfMask is hardwired to 1)    -   (b) Description: It specifies, for masked load/store operations,        which bytes to load/store and in which (consecutive) registers.        Software will guarantee that a stream only access its        corresponding context-specific mask.

FIG. 23 illustrates the ContextSpecificPAtternMAtching maskconfiguration register.

-   12. FirstInputQueue (5)    -   (a) Default Value: 0    -   (b) Description: Specifies the smallest number of the queue into        which packets from the PMMU will be inserted.-   13. SoftwareOwned (4)    -   (a) Default Value: 0 (not software owned)    -   (b) Description: one bit per LPM block. If ‘1’, the block is        software owned, which implies that the memory of the block is        managed by software, and that the VirtualPageEnable bits for        that block are a don't care.-   14. MaxActivePackets (32×9)    -   (a) Default Value: 256 for each of the queues.    -   (b) Description: Specifies, for each queue q, a value between 0        and 256 that corresponds to the maximum number of packets within        queue q that can be being processed by the SPU.

FIG. 24 illustrates the MaxActivePackets configuration register.

-   15. CodeEntryPoint (32×30)    -   (a) Default Value: 0 for each of the queues.    -   (b) Description: The contents of the CodeEntryPoint register        associated to queue q are sent to the SPU when a context is        activated which has been pre-loaded with a packet that resides        in queue q.-   16. CodeEntryPointSpecial (30)    -   (a) Default Value: 0    -   (b) Description: The contents of this register are sent to the        SPU when a context is activated due to the fact that all the        contexts become PMU-owned.-   17. Bypass Hooks (9)    -   (a) Default Value: 0    -   (b) Description: See FIG. 32. Each bit activates one hardware        bypass hook. The bypass hook is applied for as many cycles as        the corresponding bit in this register is asserted.-   18. InternalStateWrite (12)    -   (a) Default Value: 0    -   (b) Description: See FIG. 33. Specifies one word of internal PMU        state. The word of internal state will be available to software        when reading the InternalStateRead configuration register. The        InternalStateWrite configuration register is only used in one        embodiment to debug the PMU.        Read-Only Registers-   1. SizeOfOverflowedPacket (16)    -   (a) Default Value: 0    -   (b) Description: Whenever the PMU has to overflow a packet, this        register will contain the size in bytes of that packet.-   2. TimeCounter (64)    -   (a) Default Value: 0    -   (b) Description: Contains the number of core clock cycles since        the last reset of the PMU.

The TimeCounter configuration register is illustrated in FIG. 25.

-   3. StatusRegister (8)    -   (a) Default Value: 1    -   (b) Description: Contains the state of the PMU. This register is        polled by the SPU to figure out when the reset or freeze has        completed (Freeze and Reset bits), or to figure out the source        of packet error per inbound device identifier (Err: 1—error,        0—no error; EPM: 1—error has occurred while packet is overflowed        to EPM, 0—error has occurred while packet is being stored in        LPM; PSM: 1—error due to a packet size mismatch, 0—error due to        a bus error).

FIG. 26 illustrates the StatusRegister configuration register

Interrupts

The PMU can interrupt the SPU when certain events happen. Software candisable all these interrupts using some of the configuration registerslisted above. Moreover, each stream can individually mask theseinterrupts, which is the subject of a separate patent application. Thelist of interrupts that the PMU generate are as follows:

-   1. OverflowStartedInt    -   (a) Interrupt Condition: When the PMMU cannot store the incoming        packet into the LocalPacketMemory, it will overflow the packet        to the ExternalPacketMemory through the SIU.    -   (b) Disable Condition: OverflowEnable=‘0’-   2. NoMorePagesOfXSizeInt    -   (a) Interrupt Condition: When no more free virtual pages of the        size indicated in IntIfNoMoreXSizePages are available.    -   (b) Disable Condition: IntIfNoMoreXSizePages={10, 11, 12, 13,        14, 15}.-   3. LessThanXPacketIdEntriesInt    -   (a) Interrupt Condition: When the actual number of available        entries in the QS is less than IntIfLessThanXPacketIdEntries.    -   (b) Disable Condition: IntIfLessThanXPacketIdEntries=0-   4. PacketAvailableButNoContextPriorityPint (P=0 . . . 7)    -   (a) Interrupt Condition: When a packet identifier is received by        the RTU from the QS but there is no available context.    -   (b) Disable Condition:        PacketAvailableButNoContextPriorityPIntEnable=‘0’-   5. AutomaticPacketDropInt    -   (a) Interrupt Condition: When a packet cannot be stored in LPM        and OverflowEnable=‘0’.    -   (b) Disable Condition: AutomaticPacketDropIntEnable=‘0’-   6. PacketErrorInt    -   (a) Interrupt Condition: When the actual size of the packet        received from the ASIC does not match the value in the first two        bytes of the ASIC-specific header, or when a bus error has        occurred.    -   (b) Disable Condition: PacketErrorIntEnable=‘0’

Interrupts to the SPU in this embodiment are edge-triggered, which meansthat the condition that caused the interrupt is cleared in hardware whenthe interrupt is serviced. This also implies that the condition thatcauses the interrupt may happen several times before the interrupt isserved by the SPU. Therefore, the corresponding interrupt serviceroutine will be executed only once, even though the condition thatcauses the interrupt has happened more than once.

This behavior is not desirable for some of the interrupts. For thesecases, a special interlock mechanism is implemented in hardware thatguarantees that the condition will not happen again until the interrupthas been serviced.

An example of the special interlock mechanism is the case of theOverflowStartedInt and PacketAvailableButNoContextPriorityPIntinterrupts. In the first case, when a packet is overflowed, no otherpacket are overflowed until the software writes a new address in theon-the-fly configuration register OverflowAddress. If a packet has beenoverflowed but the OverflowAddress register still has not been writtenby the software, any subsequent packet that would have otherwise beenoverflowed because it does not fit in the LPM must be dropped.

For the 8 PacketAvailableButNoContextPriorityPInt (P=0 . . . 7)interrupts, the PMU architecture implicitly guarantees that no multipleconditions (per each P) will occur. This is guaranteed by design since:

-   -   (a) the PacketAvailableButNoContextPriorityPInt interrupt is        only generated when a packet identifier of RTU priority P        arrives to the RTU, and    -   (b) at most, only one packet identifier with RTU priority P        resides in the RTU.

The other interrupts can suffer from the multiple condition effect.Therefore, software should not rely on counting the number of times agiven type of interrupt happens to figure out exactly how many timesthat condition has occurred.

Protection Issues

The architecture of the PMU in the instant embodiment creates thefollowing protection issues:

-   1. An stream could read/write data from a packet other than the one    it is processing. An stream has access to all the packet memory, and    there is no mechanism to prevent an stream from accessing data from    a totally unrelated packet unless the packet memory is mapped as    kernel space.-   2. Since the configuration registers are memory mapped, any stream    could update a configuration register, no matter whether the SPU is    in single-stream mode or not. In particular, any stream could freeze    and reset the PMU.-   3. Whenever a packet is completed or moved with reactivation,    nothing prevents software from continuing “processing” the packet.    Command Unit (CU)

Software can update some information that the PMU maintains for a givenpacket and obtain this information. This is accomplished by softwarethrough some of the new XStream packet instructions referred to above.Some of these instructions are load-like in the sense that a response isrequired from the PMU. Others are store-like instructions, and noresponse is required from the PMU.

FIG. 27 is a diagram of Command Unit 213 of FIG. 2, in relation to otherblocks of the XCaliber processor in this example, all of which bear thesame element numbers in FIG. 27 as in FIG. 2. The SPU dispatches, atmost, two packet instructions per cycle across all contexts (oneinstruction per cluster of the SPU). The type of the packet instructioncorresponds to the PMU block to which the instruction affects (PMMU, QSor RTU). When the SPU dispatches a packet instruction, a single commandto the PMU is generated and inserted into one of three different queuesin the CU block (one queue per PMU block to which the command goes).Commands to the PMU are issued to PMMU command queue 2703, those to theQS go to QS command queue 2705, and command to the RTU go to the RTUcommand queue 2707. Each queue can hold up to 8 commands. The SPU onlydispatches a command to the CU if there are enough free entries in thecorresponding queue.

The CU is responsible for dispatching the commands to the respectiveblocks, and gathering the responses (if any) in an 8-entry ResponseQueue2709, which queues responses to be returned to the SPU. The CU canreceive up to three responses in a given cycle (one from each of thethree blocks). Since (a) only one outstanding packet instruction isallowed per stream, (b) the Response Queue has as many entries asstreams, (c) only one command to the PMU is generated per packetinstruction, and (d) only one response is generated per each load-likecommand, it is guaranteed that there will be enough space in theResponseQueue to enqueue the responses generated by the PMU blocks. TheResponseQueue should be able to enqueue up to two commands at a time.

CU 213 also receives requests from SIU 107 to update the configurationregisters. These commands are also sent to the PMMU, RTU and QS blocksas commands. The PMMU, QS, and RTU keep a local copy of theconfiguration registers that apply to them. The CU keeps a copy as wellof all the configuration registers, and this copy is used to satisfy theconfiguration register reads from the SIU.

For read-only configuration registers, a special interface is providedbetween the CU and the particular unit that owns the read-onlyconfiguration register. In XCaliber's PMU, there exists two read-onlyconfiguration registers: one in the PMMU block (SizeOfOverflowedPacket)and the other one in the CU block (StatusRegister). Whenever the PMMUwrites into the SizeOfOverflowedPacket register, it notifies the CU andthe CU updates its local copy.

Commands in different queues are independent and can be executed out oforder by the PMU. Within a queue, however, commands are executed inorder, and one at a time. The PMU can initiate the execution of up to 3commands per cycle. The PMMU and QS blocks give more priority to otherevents (like the creation of a new packetPage when a new packet arrives-PMMU-, or the extraction of a packet identifier because it needs to besent out -QS-) than to the commands from the SPU. This means that acommand that requests some data to be sent back to the SPU may takeseveral cycles to execute because either the PMMU or QS might be busyexecuting other operations.

RTU 227 has two sources of commands: from the QS (to pre-load packetinformation into an available context) and from the SPU (softwarecommand). The RTU always gives more priority to SPU commands. However,the RTU finishes the on-going context pre-load operation beforeexecuting the pending SPU command.

Command/Response Formats

A command received by the CMU has three fields in the currentembodiment:

-   1. Context number, which is the context associated to the stream    that generated the command.-   2. Command opcode, which is a number that specifies the type of    command to be executed by the PMU.-   3. Command data, which is the different information needed by the    PMU to execute the command specified in the command opcode field.

The PMU, upon receiving a command, determines to which of the commandqueues the command needs to be inserted. A command inserted in any ofthe queues has a similar structure as the command received, but the bitwidth of the opcode and the data will vary depending on the queue. Thetable of FIG. 28 shows the format of the command inserted in each of thequeues. Not included are the Read Configuration Register and WriteConfiguration Register commands that the CU sends to the PMMU, QS andRTU blocks.

Each command that requires a response is tagged with a number thatcorresponds to the context associated to the stream that generated thecommand. The response that is generated is also tagged with the samecontext number so that the SPU knows to which of the commands issued itbelongs.

As described above, there is only one ResponseQueue 2709 (FIG. 27) thatbuffers responses from the three PMU blocks. Note that there is no needto indicate from which block the response comes since, at most, onepacket instruction that requires a response will be outstanding perstream. Therefore, the context number associated to a response is enoughinformation to associate a response to a stream.

FIG. 29 is a table showing the format for the responses that thedifferent blocks generate back to the CU. Not included in the table arethe configuration register values provided by each of the blocks to theCU when CU performs a configuration register read.

The RTU notifies the SPU, through a dedicated interface that bypassesthe CU (path 2711 in FIG. 27), of the following events:

-   1. A masked load/store operation has finished. The interface    provides the context number.-   2. A GetContext has completed. The context number associated to the    stream that dispatched the GetContext operation, and the context    number selected by the RTU is provided by the interface. A success    bit is asserted when the GetContext succeeded; otherwise it is    de-asserted.-   3. A pre-load either starts or ends. The context number and the    priority associated to the packet is provided to the SPU.    Reset and Freeze Modes

The PMU can enter the reset mode in two cases:

-   1. SPU sets the Reset configuration flag.-   2. XCaliber is booted.

The PMU can also enter the freeze mode in two cases:

-   1. SPU sets the Freeze configuration flag.-   2. PMU finishes the reset sequence.

The reset sequence of the PMU takes several cycles. During thissequence, the Reset bit in the StatusRegister configuration register isset. After the reset sequence, all the configuration registers are setto their default values, and the PMU enters the freeze mode (the Resetbit in the StatusRegister is reset and the Freeze bit is set). When thisis done, the SPU resets the Freeze configuration flag and, from thattime on, the PMU runs in the normal mode.

When the SPU sets the Freeze configuration flag, the PMU terminates thecurrent transaction or transactions before setting the Freeze bit in theStatusRegister. Once in the freeze mode, the PMU will not accept anydata from the network input interface, send any data out through thenetwork output interface, or pre-load any packet

The PMU continues executing all the SPU commands while in freeze mode.

The SPU needs to poll the StatusRegister configuration register todetermine in which mode the PMU happened to be (reset or freeze) and todetect when the PMU changes modes.

The CU block instructs the rest of the blocks to perform the reset andthe freeze. The following is the protocol between the CU and any otherblock when the CU receives a write into the reset and/or freezeconfiguration bit:

-   1. The CU notifies to some of the blocks that either a freeze or a    reset needs to be performed.-   2. Every block performs the freeze or the reset. After completion,    the block signals back to the CU that it has completed the freeze or    reset.-   3. The CU updates the StatusRegister bits as soon as the reset or    freeze has been completed. Software polls the StatusRegister to    determine when the PMU has completely frozen.    The different blocks in the PMU end the freeze when:-   1. IB, LPM, CU and QS do not need to freeze.-   2. As soon as the PMMU finishes uploading inbound packets, if any,    and downloading outbound packets, if any.-   3. As soon as the RTU has finished the current pre-load operation,    if any.-   4. As soon as the OB is empty.    While in freeze mode, the blocks will not:-   1. start uploading a new packet; start downloading a completed    packet; or generate interrupts to the SPU (PMMU)-   2. pre-load a context or generate interrupts to the SPU (RTU).

If software writes a ‘1’ in the Freeze/Reset configuration register andthen writes a ‘0’ before the PMU froze or reset, results are undefined.Once the PMU starts the freeze/reset sequence, it completes it.

Performance Counters Interface

The PMU probes some events in the different units. These probes are sentto the SIU and used by software as performance probes. The SIU has a setof counters used to count some of the events that the PMU send to theSIU. Software decides which events throughout the XCaliber chip it wantsto monitor. Refer to the SIU Architecture Spec document for moreinformation on how software can configure the performance counters.

FIG. 30 shows a performance counter interface between the PMU and theSIU. Up to 64 events can be probed within the PMU. All 64 events aresent every cycle to the SIU (EventVector) through a 64-bit bus.

Each of the 64 events may have associated a value (0 to 64K-1). Softwareselects two of the events (EventA and EventB). For each of these two,the PMU provides the associated 16-bit value (EventDataA and EventDataB,respectively) at the same time the event is provided in the EventVectorbus.

Events are level-triggered. Therefore, if the PMU asserts the event fortwo consecutive cycles, the event will be counted twice. Thecorresponding signal in the EventVector will be asserted only if theevent occurs, and for as many cycles as the event condition holds.

The SIU selects which events are actually counted (based on how softwarehas programmed the SIU). If the SIU decides to count an event numberdifferent from EventA or EventB, a counter within the SIU counts theevent for as many cycles the corresponding bit in the EventVector isasserted. If the events monitored are EventA and/or EventB, the SIU, inaddition to counting the event/s, increments another counter byEventDataA and/or EventDataB every time the event occurs.

FIG. 31 shows a possible implementation of the internal interfaces amongthe different blocks in PMU 103. CU acts as the interface between thePMU and SIU for the performance counters. CU 213 distributes theinformation in EventA and EventB to the different units and gathers theindividual EventVector, EventDataA and EventDataB of each of the units.

The CU block collects all the events from the different blocks and sendthem to the SIU. The CU interfaces to the different blocks to notifywhich of the events within each block need to provide the EventDataAand/or EventDataB values.

Performance events are not time critical, i.e. they do not need to bereported to the SIU in the same cycle they occur.

FIGS. 34 through 39 comprise a table that lists all events related toperformance counters. These events are grouped by block in the PMU. Theevent number is shown in the second column. This number corresponds tothe bit in the EventVector that is asserted when the event occurs. Thethird column is the event name. The fourth column shows the data valueassociated to the event and its bit width in parentheses. The lastcolumn provides a description of the event.

The CU block collects all of the events from the different blocks andsends them to the SIU. The CU interfaces to the different blocks tonotify which of the events within each block need to provide theEventDataA and the EventDataB values.

Performance events are not time critical, i.e. they do not need to bereported to the SIU in the same cycle that they occur.

Debug Bypasses and Trigger Events

Hardware debug hooks are implemented in the PMU to help debugging of thesilicon. The debug hooks are divided into two categories:

-   1. Bypass hooks: will bypass potentially faulty functions. Instead    of the faulty results generated by these functions (or, in some    cases, no result at all), the bypass hook will provide at least some    functionality that will allow other neighboring blocks to be tested.-   2. Trigger events: when a particular condition occurs in the PMU    (trigger event), the PMU will enter automatically in single-step    mode until, through the OCI Interface (Section), the SIU sends a    command to the PMU to exit the single-step mode.

Moreover, the PMU has the capability of being single-stepped. A signal(SingleStep) will come from the OCI Interface. On a cycle-by-cyclebasis, the different blocks of the PMU will monitor this signal. Whenthis signal is de-asserted, the PMU will function normally. WhenSingleStep is asserted, the PMU will not perform any work: any operationon progress will be held until the signal is de-asserted. In otherwords, the PMU will not do anything when the signal is asserted. Theonly exception to this is when a block can lose data (an example couldbe in the interface between two block: a block A sends data to a block Band assumes that block B will get the data in the next cycle; ifSingleStep is asserted in this cycle, block B has to guarantee that thedata from A is not lost).

Bypass Hooks

The different bypass hooks in the PMU are activated through theon-the-fly BypassHooks configuration register. FIG. 40 is a tableillustrating the different bypass hooks implemented in the PMU. Thenumber of each hook corresponds to the bit number in the BypassHooksregister. The bypass hook is applied for as many cycles as thecorresponding bit in this register is asserted.

Trigger Events

The following is a list of trigger events implemented in the PMU.

-   1. A new packet of size s bytes is at the head of the IBU.    -   (a) s=0: any packet.-   2. A packetId from source s with packetPage pp is inserted in queue    q in the QS.    -   (a) s=0: PMM, s=1: QS, s=2: CMU; s=3: any    -   (b) pp=0x10000: any    -   (c) q=33: any-   3. A packetId from queue q with packetPage pp and packetNumber pn is    sent to RTU.    -   (a) pp=0x10000: any    -   (b) q=33: any    -   (c) pn=256: any-   4. A packetId with packetPage pp and packetNumber pn reaches the    head of queue q in the QS.    -   (a) pp=0x10000: any    -   (b) q=33: any    -   (c) pn=256: any-   5. A packet with RTU priority p and packetPage pp and packetNumber    pn is pre-loaded in context c.    -   (a) pp=0x10000: any    -   (b) q=33: any    -   (c) pn=256: any    -   (d) c=8: any-   6. A packetId from queue q with packetPage pp and packetNumber pn is    sent for downloading to PMM.    -   (a) pp=0x10000: any    -   (b) q=33: any    -   (c) pn=256: any-   7. A packetId with packetPage pp and packetNumber pn reaches the    head of queue q in the QS.    -   (a) pp=0x10000: any    -   (b) q=33: any    -   (c) pn=256: any-   8. Packet command pc is executed by block b.    -   (a) pc=0: GetSpace; pc=1: FreeSpace; pc=2: InsertPacket; pc=3:        ProbePacket; pc=4: ExtractPacket; pc=5: CompletePacket; pc=6:        UpdatePacket; pc=7: MovePacket; pc=8: ProbeQueue; pc=9:        GetContext; pc=10: ReleaseContext; pc=11: MaskedLoad; pc=12:        MaskedStore; pc=13: any    -   (b) b=0: RTU; b=1: PMM; b=2: QS; b=3: any        Detailed Interfaces with the SPU and SIU

The architecture explained in the previous sections is implemented inthe hardware blocks shown in FIG. 41:

SPU-PMU Interface

FIGS. 42-45 describe the SPU-PMU Interface.

SIU-PMU Interface

FIGS. 46-49 describe the SIU-PMU Interface.

The specification above describes in enabling detail a Packet MemoryUnit (PMU) for a Multi-Streaming processor adapted for packet handlingand processing. Details of architecture, hardware, software, andoperation are provided in exemplary embodiments. It will be apparent tothe skilled artisan that the embodiments described may vary considerablyin detail without departing from the spirit and scope of the invention.It is well-known, for example, that IC hardware, firmware and softwaremay be accomplished in a variety of ways while still adhering to thenovel architecture and functionality taught.

Non-Speculative Pre-Fetch Operation

In another aspect of the present invention, the inventor provides amethod and apparatus that enables a non-speculative pre-fetch ofprocessing instructions performed by the SPU upon early notificationfrom the PMU that a context has been selected for processing and thatpreloading of the context will begin. Such a method and apparatus isdescribed in enabling detail below.

FIG. 50 is a block diagram illustrating various elements and interactionbetween elements in performance of a non-speculative pre-fetch operationaccording to one embodiment of the present invention.

Referring to Ser. No. 09/737,375 listed as a priority document in thecross-reference section above, there is disclosed a general method forselecting a context, pre-loading the context with packet information,and then releasing the context to the SPU for processing. The headingsunder which the disclosure is made are Register Transfer Unit (RTU),Context States and Pre-loading a Context.

Because a context being pre-loaded for packet processing is always aPMU-owned context, the RTU has all the available write ports to thatcontext to perform the loading of packet information. It is disclosedabove under the heading Pre-loading a Context that whenever thepre-loading operation starts, the RTU notifies this event to the SPUthrough a dedicated interface. Similarly, when the pre-loading operationis completed, the RTU also notifies the SPU of this fact. Thus the SPUexpects two basic notifications (start and end) for each packet pre-loadoperation. A special notification is provided to the SPU when the RTUstarts and ends a pre-load in a same cycle.

In the instant example referring to FIG. 50, a packet management unit(PMU) 5102 is provided having a register transfer unit (RTU) 5103illustrated therein, the RTU having a software-configurable hardwaretable (T) 5104 available thereto. PMU 5102, more specifically throughRTU 5103, has a dedicated communication link 5106 established betweenitself and streaming processor unit (SPU) 5107. SPU 5107 is adapted toprocess the packets using instructions that are pre-fetched inembodiments of the present invention.

SPU 5107, in a preferred embodiment, is connected to an instructioncache memory 5109, which is adapted to store, among other data, firstinstructions of threads for processing data, and in some casessequential instructions for specific threads. Connection from SPU 5107to cache 5109 is logically represented herein by a link 5108. Storingthe first instruction of a thread in an on-chip instruction cache is notrequired for the invention, as the first instruction can be anywhere inmemory, even on a disk, but it is a convenience and preferred to havethe instructions stored as close as possible to the processing core. Inan embodiment of the present invention packets arriving for processingare staged in queues according to packet types, and a specific thread isassociated with each packet type for processing. In this embodiment ofthe invention a table 5104 associates queues (packet types) to specificthreads needed for processing by a program counter (PC) pointer,indicated in FIG. 50 as PC# and element 5105. PC# 5105 is not to beconfused with a packet command (pc) disclosed with reference to Ser. No.09/737,375 under the heading Trigger Events.

A cluster 5101 of contexts and functional resources generic to theprocessing core of SPU 5107 is illustrated in this example. Functionalresources are circuitry required to perform calculations such asmultiplication, division, addition and subtraction. There may also bespecial functions such as trigonometric, averaging, and weightingfunctions performed by functional units, and memory access functions aswell. Contexts are well known in the art, and are register files intowhich, in this case, packet information is loaded prior to processing.The illustration of contexts and resources is exemplary only in thisexample, as there may be different numbers of each, and there isgenerally not a one-to-one correspondence between resources andfunctional units.

It is the responsibility of PMU 5102 through RTU 5103 via link 5106 toselect available (not SPU-owned) contexts in cluster 5101 for preloadingpacket information thereto for processing by SPU-5107 and to activatethose selected and loaded contexts, at which time SPU 5107 will own theactivated contexts until processing is complete. It is noted herein thatany context in cluster 5101 may issue instructions only to thefunctional resources within that cluster. When SPU 5107 finishesprocessing a thread it releases the context back to PMU 5102.

Table 5104 contains a PC#5105 for each of the different queues intowhich packets can be classified by the PMU. PC#5105 represents, in apreferred embodiment, the cache memory address of the beginning of itscorresponding thread. In other embodiments the PC may point to anaddress for a first instruction for a thread in any memory deviceavailable to the processing core. It is noted herein that in a preferredembodiment there are 32 queues available for storing identifiers of datapackets. The number of 32 is not meant to be a limitation, as therecould be more or less than 32 queues provided and made available invarious configurations. In practice of the invention a packet arrivesfor processing and is en-queued into one of the 32 available queuesaccording to a classification scheme which may include priority. Thescheme in a preferred embodiment revolves around packet type. Forexample, a voice-over-Internet protocol (VoIP) packet may be assigned ahigher priority than an e-mail packet. Therefore, the VoIP packet willbe en-queued in one of the 32 queues of higher number, perhaps queue 32if VoIP packets are assigned the highest priority in a particularscheme, which may be varied according to enterprise design. In fact,there are more than one type of VoIP packets that may be encountered andthey may differ somewhat from each other in exact instruction typesrequired to process them. Therefore, there may be more than one queuededicated for VoIP packets of differing types. It may be that queuenumbers 29-32 are dedicated to the range of VoIP packets encountered.Other types of data packets encountered are similarly queued accordingto type and priority level of processing.

Each queue has associated with it a corresponding PC# 5105. When RTU5103 selects an available context from within cluster 5101 forprocessing a newly arrived data packet, it sends a notification of thefact to SPU 5107 over dedicated link 5106. This notification containsthe correct PC# (associated to the queue) for that queue. The PC#identifies the beginning memory location or address of the appropriatethread for processing stored in instruction cache 5109 in a preferredembodiment, or the address for a first instruction for a thread in anyother memory device in alternative embodiments. The SPU will be able tofetch and execute from that point on all of the instructions of whichthe thread is composed. A thread ends with a special instruction (calledrelease) that effectively send the ownership of the context the streamis running, back to the PMU by notification via link 5106.

Immediately after the first notification that a context has beenselected for processing, RTU 5103 begins loading packet information intothe selected context for processing. Simultaneously, in this embodiment,SPU 5107 fetches the appropriate instruction thread from cache 5109 overlink 5108 using PC# 5105 as a pointer. After loading the selectedcontext with the appropriate data for processing, RTU 5103 sends anotification of activation of the context to SPU 5107 over link 5106.SPU 5107, assuming that it has completed the pre-fetch, may thencommence processing. In some cases, particularly those cases in whichthere is no instruction cache, and the thread must be fetched from amore distant memory, the pre-fetch may take longer than the loading ofthe context. In one embodiment, a special packet identification threadis provided to handle a possible situation wherein a packet sender doesnot include information designating the type of data packet and/or theappropriate queue destination. In this case, the unidentified packet isen-queued into a general queue set aside for this purpose. This generalqueue has a PC# associated therewith and included in table (T) 5104.Thus when a context is selected for processing the packet by RTU 5103,the notification to SPU contains the PC# pointing to the specialpacket-type identification instruction (the start of the thread) storedin cache 5109, in a preferred embodiment. The SPU pre-fetches thespecial thread as described in the normal sequence above. Duringprocessing, the special thread will determine the exact packet type andthe appropriate queue that it should be placed in. At this time thepacket is re-queued in the appropriate queue, after which a new contextis subsequently selected and re-notification to SPU 5107 is initiated,or the special thread might decide to process the packet itself.

The special circumstance described above needs only be performed on afirst data packet of a data packet flow from a same source. Thedetermined classification information (appropriate queue for packets ofthis flow) can be tabled by SPU 5107 within T 5104 and sent back to RTU5103 so that a next packet of the same flow can be properly classifiedand information about the packet can be en-queued in the appropriatequeue. If for example, the unclassified packet was determined to be atype not accounted for in terms of existing instruction threads, then adetermination may be made to create a new thread, assign a PC# and queueto handle the new packet type. In still another embodiment, a specialhardware mechanism is provided at the port for intercepting unclassifieddata packets. The hardware has its own queue and associated PC# and isenhanced with a processing capability and functional resources to atleast identify the packet independently from the SPU. After the packetis classified by the hardware, it is looped back to ingress for properqueuing according to priority.

Referring to the first and preferred embodiment described above, it maybe that SPU 5107, while processing an unclassified packet foridentification, will find that the determined priority of the packet isnot high and that there are numerous packets waiting for processing thatare classified and of a higher priority. In this case, an interrupt isgenerated to cease processing the packet and release the context back tothe PMU without re-queuing the packet. The packet can remain in thegeneral queue until the higher priority packets are processed. This, ofcourse assumes that the SPU has knowledge of multiple-queued packetsbefore processing, information which can be propagated over link 5106from RTU 5102 within T 5104.

Because all packets are queued by type, and each queue is associatedwith a unique PC# pointing to an address for the beginning of anappropriate thread stored in cache 5109, or in another memory device,SPU 5108 is enabled to perform a non-speculative pre-fetch, thus assuredthat the instructions retrieved are the actual instructions required forprocessing.

FIG. 51 is a process flow chart illustrating steps for implementing anon-speculative pre-fetch operation according to an embodiment of thepresent invention. At step 5201 a data packet arrives for processing. Aspreviously described above there are 32 available queues in a preferredembodiment wherein information pertinent to the data packet may beplaced according to class and priority. Each queue has an associatedprogram counter (PC#) that points to an address in memory for thebeginning of a specific thread required to process the data packet.

Time is indicated in FIG. 51 on a vertical axis, activities of the RTUare indicated on the left of the figure, and activities of the SPU areindicated on the right of the figure. At step 5202, the RTU selects anavailable (not SPU-owned) context and notifies the SPU that theparticular context will be activated for processing. In the notificationat step 5202, the PC# is provided from the association with the queuefor the packet, indicating the address for the first instruction for thethread to process the packet.

Upon receipt of the notification from the RTU at step 5202, the SPU maybegin pre-fetching the appropriate thread. At substantially the sametime the RTU, at step 5203, begins loading packet information into theselected context. At step 5205, loading is complete, and the RTUnotifies the SPU and releases (activates) the context.

There are two necessary conditions for the SPU to process the data inthe context. One is that the RTU releases the context, and notifies theSPU. The other is that the SPU has loaded the first instruction of anappropriate thread for processing. Either condition may finish first,so, in one case the SPU will wait for the RTU, and begin processing assoon as the release notification arrives from the RTU; while in theother case the SPU will receive the notification from the RTU, but willfinish loading the appropriate thread before beginning to process atstep 5208.

The optional situation is indicated in FIG. 51 by alternate paths forthe SPU. IN one case the SPU finishes pre-fetch at step 5206 before theRTU finishes loading, and the SPU must therefore wait for the loading tofinish, and for the notification from the RTU before processing maycommence. In the other option, shown as step 5207, the notification fromthe RTU arrives before the SPU finishes pre-fetch, so the SPU continues,and processing may commence at step 5207 when the pre-fetch is finished.

The present invention is particularly applicable to the processing ofdata packets by data packet routers connected to a data packet network.However, this should not be construed as a limitation of the presentinvention. Other types of data processing machines such as Internet dataservers, e-mail servers, and so on may benefit from the presentinvention. Accordingly the claims that follow should be accorded thebroadest interpretation. The spirit and scope of the present inventionis limited only by the claims that follow.

1. A system for processing packets, the system comprising: a packetmanagement unit (PMU) to: select a context for processing a data packet;identify a thread for use in processing the data packet; convey anidentification of the thread and the context; and load informationcorresponding to the packet into the context; and a processing unit to:receive the identification of the thread and the context from the PMU;and initiate a pre-fetch of instructions of the thread prior to the PMUcompleting the load of the information into the context.
 2. The systemof claim 1, wherein the packet management unit is identifies said threadby accessing a table which associates threads with packet types.
 3. Thesystem of claim 2, wherein said table comprises a plurality of queues,each of said queues for storing packets of a particular type.
 4. Thesystem of claim 3, wherein for each of said queues, the table comprisesan identifier which identifies a thread for use in processing packets ofa corresponding type.
 5. The system of claim 4, wherein said identifiercomprises an address of a first instruction of a thread.
 6. The systemof claim 2, wherein the table comprises a software configurable hardwaretable.
 7. The system of claim 1, wherein said processing unit comprisesa first portion for processing the data packets, and a second portioncomprising a pool of context registers.
 8. The system of claim 7,wherein said PMU selects said context from said pool of contexts via adedicated link which couples the PMU to the processing unit.
 9. Thesystem of claim 1, wherein the processing unit pre-fetches one or moreof said instructions at least partially in parallel with the PMU loadingthe information corresponding to the packet into the context.
 10. Thesystem of claim 3, wherein the PMU further en-queues a received datapacket into one of said queues according to a classification scheme. 11.The system of claim 10, wherein the classification scheme is based onpacket type and/or priority.
 12. The system of claim 11, wherein saidpacket types include voice over Internet protocol packets and/or e-mailpackets.
 13. The system of claim 3, wherein in response to receiving agiven packet of an undetermined type, the PMU stores the given packet ina special queue dedicated for storing unclassified packets, and conveyan identification of a thread configured to identify packet types. 14.The system of claim 1, wherein said PMU and processing unit are part ofa dynamic multi-streaming processor.
 15. The system as recited in claim1, wherein said pre-fetch is non-speculative.
 16. A method forprocessing data packets, the method comprising: selecting a context forprocessing a data packet; identifying a thread for use in processing thedata packet; conveying an identification of the thread and the context;loading information corresponding to the packet into the context; andinitiating a pre-fetch of instructions of the thread prior to completingsaid loading of the information into the context.
 17. The method ofclaim 16, wherein said identifying comprises accessing a table whichassociates threads with packet types.
 18. The method of claim 17,wherein said table comprises a plurality of queues, each of said queuesbeing configured to store packets of a particular type.
 19. The methodof claim 18, wherein for each of said queues, the table comprises anidentifier which identifies a thread for use in processing packets of acorresponding type.
 20. The method of claim 19, wherein said identifiercomprises an address of a first instruction of a thread.
 21. The methodof claim 17, wherein the table comprises a software configurablehardware table.
 22. The method of claim 21, wherein said methodcomprises selecting said context from a pool of contexts.
 23. The methodof claim 16, wherein pre-fetching said instructions is performed atleast partially in parallel with the loading of the informationcorresponding to the packet into the context.
 24. The method of claim18, further comprising en-queuing a received data packet into one ofsaid queues according to a classification scheme.
 25. The method ofclaim 24, wherein the classification scheme is based on packet typeand/or priority.
 26. The method of claim 25, wherein said packet typesinclude voice over Internet protocol packets and/or e-mail packets. 27.The method of claim 18, wherein in response to receiving a given packetof an undetermined type, the method comprises storing the given packetin a special queue dedicated for storing unclassified packets, andconveying an identification of a thread configured to identify packettypes.
 28. The method of claim 16, wherein said pre-fetch isnon-speculative.